2024-09-18T00:43:46.043 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-18T00:43:46.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T00:43:46.134 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909680' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: rbd cache: true rbd clone copy on read: true global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd memory target: 1610612736 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1983 sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f sleep_before_teardown: 0 subset: 79/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDdmUOl4ryUlfP/sCj0rAvvbzPuu4kPaUquCAH01QG7QHP+G5eLIdDDp+yQj9z4KKmsqR1Wd3X5HqxafhC6b5mo= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAXWVER4b9q8kCr3qUCuRi6Qv0nTyEZlr8rhOHtyTWFOLkj+watrKaSAZQYUG526cGmaEgMv28zbrD3BXosoroI= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-18T00:43:46.136 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa; will attempt to use it 2024-09-18T00:43:46.137 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa/tasks 2024-09-18T00:43:46.137 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-18T00:43:46.138 INFO:teuthology.task.internal:Checking packages... 2024-09-18T00:43:46.159 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'c2ddf9721ef254645201cdbabd9c8f49c30a586f' 2024-09-18T00:43:46.159 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-18T00:43:46.159 INFO:teuthology.packaging:ref: None 2024-09-18T00:43:46.160 INFO:teuthology.packaging:tag: None 2024-09-18T00:43:46.160 INFO:teuthology.packaging:branch: squid 2024-09-18T00:43:46.160 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:43:46.160 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-18T00:43:46.385 INFO:teuthology.task.internal:Found packages for ceph version 19.2.0-1jammy 2024-09-18T00:43:46.387 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-18T00:43:46.392 INFO:teuthology.task.internal:no buildpackages task found 2024-09-18T00:43:46.393 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-18T00:43:46.397 INFO:teuthology.task.internal:Saving configuration 2024-09-18T00:43:46.410 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-18T00:43:46.416 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-18T00:43:46.440 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 00:38:08.176131', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDK783Uyxo2AHNxNv3pxFftlDtRWPB+HZjYa23kQf06x'} 2024-09-18T00:43:46.462 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 00:38:08.177655', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEawJTrmY2U+l8rBisUJgVMkRU5BgGjh5XwQjcHgzb38'} 2024-09-18T00:43:46.462 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-18T00:43:46.468 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-18T00:43:46.468 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-18T00:43:46.469 INFO:teuthology.run_tasks:Running task console_log... 2024-09-18T00:43:46.579 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f26bd5af400>, signals=[15]) 2024-09-18T00:43:46.580 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-18T00:43:46.585 INFO:teuthology.task.internal:Opening connections... 2024-09-18T00:43:46.585 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-09-18T00:43:46.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:43:46.665 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-09-18T00:43:46.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:43:46.738 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-18T00:43:46.744 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-09-18T00:43:46.749 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-09-18T00:43:46.752 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-09-18T00:43:46.795 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-18T00:43:46.795 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="Ubuntu" 2024-09-18T00:43:46.795 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="22.04" 2024-09-18T00:43:46.795 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-18T00:43:46.795 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_CODENAME=jammy 2024-09-18T00:43:46.795 INFO:teuthology.orchestra.run.smithi042.stdout:ID=ubuntu 2024-09-18T00:43:46.795 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE=debian 2024-09-18T00:43:46.795 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-18T00:43:46.795 INFO:teuthology.orchestra.run.smithi042.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-18T00:43:46.795 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-18T00:43:46.796 INFO:teuthology.orchestra.run.smithi042.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-18T00:43:46.796 INFO:teuthology.orchestra.run.smithi042.stdout:UBUNTU_CODENAME=jammy 2024-09-18T00:43:46.796 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-09-18T00:43:46.820 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-09-18T00:43:46.825 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-09-18T00:43:46.826 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-09-18T00:43:46.871 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-18T00:43:46.871 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="Ubuntu" 2024-09-18T00:43:46.871 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="22.04" 2024-09-18T00:43:46.872 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-18T00:43:46.872 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_CODENAME=jammy 2024-09-18T00:43:46.872 INFO:teuthology.orchestra.run.smithi195.stdout:ID=ubuntu 2024-09-18T00:43:46.872 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE=debian 2024-09-18T00:43:46.872 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-18T00:43:46.872 INFO:teuthology.orchestra.run.smithi195.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-18T00:43:46.872 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-18T00:43:46.872 INFO:teuthology.orchestra.run.smithi195.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-18T00:43:46.872 INFO:teuthology.orchestra.run.smithi195.stdout:UBUNTU_CODENAME=jammy 2024-09-18T00:43:46.872 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-09-18T00:43:46.895 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-18T00:43:46.902 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-18T00:43:46.907 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-18T00:43:46.907 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-09-18T00:43:46.909 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-09-18T00:43:46.919 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-18T00:43:46.924 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-18T00:43:46.924 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-09-18T00:43:46.956 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-09-18T00:43:46.976 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-18T00:43:47.031 INFO:teuthology.run_tasks:Running task kernel... 2024-09-18T00:43:47.044 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-18T00:43:47.044 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-18T00:43:47.044 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T00:43:47.045 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T00:43:47.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:43:47.049 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-09-18T00:43:47.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:43:47.050 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-09-18T00:43:47.095 INFO:teuthology.orchestra.run.smithi042.stdout:5.15.0-56-generic 2024-09-18T00:43:47.095 INFO:teuthology.task.kernel:Running kernel on smithi042: 5.15.0-56-generic 2024-09-18T00:43:47.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-get clean 2024-09-18T00:43:47.096 INFO:teuthology.orchestra.run.smithi195.stdout:5.15.0-56-generic 2024-09-18T00:43:47.097 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.15.0-56-generic 2024-09-18T00:43:47.097 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2024-09-18T00:43:47.214 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-get update 2024-09-18T00:43:47.228 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-09-18T00:43:47.350 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:43:47.364 INFO:teuthology.orchestra.run.smithi042.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-18T00:43:47.364 INFO:teuthology.orchestra.run.smithi042.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-18T00:43:47.430 INFO:teuthology.orchestra.run.smithi042.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-18T00:43:47.515 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:43:47.523 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-18T00:43:47.573 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-18T00:43:47.601 INFO:teuthology.orchestra.run.smithi042.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-18T00:43:47.682 INFO:teuthology.orchestra.run.smithi042.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-18T00:43:47.700 INFO:teuthology.orchestra.run.smithi042.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-18T00:43:47.706 INFO:teuthology.orchestra.run.smithi042.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-18T00:43:47.709 INFO:teuthology.orchestra.run.smithi042.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-18T00:43:47.710 INFO:teuthology.orchestra.run.smithi042.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-18T00:43:47.730 INFO:teuthology.orchestra.run.smithi042.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-18T00:43:47.731 INFO:teuthology.orchestra.run.smithi042.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-18T00:43:47.736 INFO:teuthology.orchestra.run.smithi042.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-18T00:43:47.736 INFO:teuthology.orchestra.run.smithi042.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-18T00:43:47.747 INFO:teuthology.orchestra.run.smithi042.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-18T00:43:47.754 INFO:teuthology.orchestra.run.smithi042.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-18T00:43:47.756 INFO:teuthology.orchestra.run.smithi042.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-18T00:43:47.756 INFO:teuthology.orchestra.run.smithi042.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-18T00:43:47.756 INFO:teuthology.orchestra.run.smithi042.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-18T00:43:47.762 INFO:teuthology.orchestra.run.smithi042.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-18T00:43:47.798 INFO:teuthology.orchestra.run.smithi042.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-18T00:43:47.798 INFO:teuthology.orchestra.run.smithi042.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-18T00:43:47.798 INFO:teuthology.orchestra.run.smithi042.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-18T00:43:47.799 INFO:teuthology.orchestra.run.smithi042.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-18T00:43:47.800 INFO:teuthology.orchestra.run.smithi042.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-18T00:43:47.800 INFO:teuthology.orchestra.run.smithi042.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-18T00:43:47.800 INFO:teuthology.orchestra.run.smithi042.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-18T00:43:47.801 INFO:teuthology.orchestra.run.smithi042.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-18T00:43:47.801 INFO:teuthology.orchestra.run.smithi042.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-18T00:43:47.801 INFO:teuthology.orchestra.run.smithi042.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-18T00:43:47.803 INFO:teuthology.orchestra.run.smithi042.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-18T00:43:47.805 INFO:teuthology.orchestra.run.smithi042.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-18T00:43:47.805 INFO:teuthology.orchestra.run.smithi042.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-18T00:43:47.806 INFO:teuthology.orchestra.run.smithi042.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-18T00:43:47.823 INFO:teuthology.orchestra.run.smithi042.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-18T00:43:47.826 INFO:teuthology.orchestra.run.smithi042.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-18T00:43:47.826 INFO:teuthology.orchestra.run.smithi042.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-18T00:43:47.835 INFO:teuthology.orchestra.run.smithi042.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-18T00:43:47.839 INFO:teuthology.orchestra.run.smithi042.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-18T00:43:47.841 INFO:teuthology.orchestra.run.smithi042.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-18T00:43:47.841 INFO:teuthology.orchestra.run.smithi042.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-18T00:43:47.841 INFO:teuthology.orchestra.run.smithi042.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-18T00:43:47.841 INFO:teuthology.orchestra.run.smithi042.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-18T00:43:47.856 INFO:teuthology.orchestra.run.smithi042.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-18T00:43:47.928 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-18T00:43:47.949 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-18T00:43:48.067 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-18T00:43:48.245 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-18T00:43:48.387 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-18T00:43:48.421 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-18T00:43:48.424 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-18T00:43:48.433 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-18T00:43:48.434 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-18T00:43:48.435 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-18T00:43:48.438 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-18T00:43:48.439 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-18T00:43:48.509 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-18T00:43:48.510 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-18T00:43:48.573 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-18T00:43:48.573 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-18T00:43:48.573 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-18T00:43:48.580 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-18T00:43:48.580 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-18T00:43:48.592 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-18T00:43:48.592 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-18T00:43:48.605 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-18T00:43:48.607 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-18T00:43:48.609 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-18T00:43:48.610 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-18T00:43:48.610 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-18T00:43:48.611 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-18T00:43:48.611 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-18T00:43:48.611 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-18T00:43:48.612 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-18T00:43:48.633 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-18T00:43:48.633 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-18T00:43:48.667 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-18T00:43:48.693 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-18T00:43:48.694 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-18T00:43:48.695 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-18T00:43:48.696 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-18T00:43:48.696 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-18T00:43:48.714 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-18T00:43:48.715 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-18T00:43:48.715 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-18T00:43:54.385 INFO:teuthology.orchestra.run.smithi042.stdout:Fetched 16.2 MB in 2s (7,089 kB/s) 2024-09-18T00:43:54.729 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 16.2 MB in 3s (5,820 kB/s) 2024-09-18T00:43:55.903 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-18T00:43:55.922 DEBUG:teuthology.orchestra.run.smithi042:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T00:43:55.980 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-18T00:43:56.152 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-18T00:43:56.170 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T00:43:56.186 INFO:teuthology.orchestra.run.smithi042.stdout:Building dependency tree... 2024-09-18T00:43:56.187 INFO:teuthology.orchestra.run.smithi042.stdout:Reading state information... 2024-09-18T00:43:56.234 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-18T00:43:56.362 INFO:teuthology.orchestra.run.smithi042.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:43:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:43:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:43:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:43:56.362 INFO:teuthology.orchestra.run.smithi042.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-18T00:43:56.363 INFO:teuthology.orchestra.run.smithi042.stdout: python2.7-minimal 2024-09-18T00:43:56.363 INFO:teuthology.orchestra.run.smithi042.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:43:56.364 INFO:teuthology.orchestra.run.smithi042.stdout:The following additional packages will be installed: 2024-09-18T00:43:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-18T00:43:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-18T00:43:56.364 INFO:teuthology.orchestra.run.smithi042.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-18T00:43:56.365 INFO:teuthology.orchestra.run.smithi042.stdout:Suggested packages: 2024-09-18T00:43:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-18T00:43:56.366 INFO:teuthology.orchestra.run.smithi042.stdout:Recommended packages: 2024-09-18T00:43:56.366 INFO:teuthology.orchestra.run.smithi042.stdout: thermald 2024-09-18T00:43:56.413 INFO:teuthology.orchestra.run.smithi042.stdout:The following NEW packages will be installed: 2024-09-18T00:43:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-18T00:43:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-18T00:43:56.414 INFO:teuthology.orchestra.run.smithi042.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-18T00:43:56.415 INFO:teuthology.orchestra.run.smithi042.stdout:The following packages will be upgraded: 2024-09-18T00:43:56.415 INFO:teuthology.orchestra.run.smithi042.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-18T00:43:56.449 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-09-18T00:43:56.449 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-09-18T00:43:56.466 INFO:teuthology.orchestra.run.smithi042.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-18T00:43:56.466 INFO:teuthology.orchestra.run.smithi042.stdout:Need to get 113 MB of archives. 2024-09-18T00:43:56.467 INFO:teuthology.orchestra.run.smithi042.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-18T00:43:56.467 INFO:teuthology.orchestra.run.smithi042.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-18T00:43:56.626 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:43:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:43:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:43:56.626 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:43:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-18T00:43:56.627 INFO:teuthology.orchestra.run.smithi195.stdout: python2.7-minimal 2024-09-18T00:43:56.627 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:43:56.628 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-09-18T00:43:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-18T00:43:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-18T00:43:56.628 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-18T00:43:56.629 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-09-18T00:43:56.630 INFO:teuthology.orchestra.run.smithi195.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-18T00:43:56.630 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2024-09-18T00:43:56.630 INFO:teuthology.orchestra.run.smithi195.stdout: thermald 2024-09-18T00:43:56.673 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-09-18T00:43:56.673 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-18T00:43:56.673 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-18T00:43:56.673 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-18T00:43:56.675 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages will be upgraded: 2024-09-18T00:43:56.675 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-18T00:43:56.856 INFO:teuthology.orchestra.run.smithi195.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-18T00:43:56.856 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 113 MB of archives. 2024-09-18T00:43:56.856 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-18T00:43:56.856 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-18T00:43:56.912 INFO:teuthology.orchestra.run.smithi042.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-18T00:43:57.024 INFO:teuthology.orchestra.run.smithi042.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-18T00:43:57.866 INFO:teuthology.orchestra.run.smithi042.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-18T00:43:57.866 INFO:teuthology.orchestra.run.smithi042.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-18T00:43:57.866 INFO:teuthology.orchestra.run.smithi042.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-18T00:43:57.997 INFO:teuthology.orchestra.run.smithi042.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-18T00:43:58.019 INFO:teuthology.orchestra.run.smithi042.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-18T00:43:58.270 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-18T00:43:58.548 INFO:teuthology.orchestra.run.smithi042.stdout:Fetched 113 MB in 2s (68.0 MB/s) 2024-09-18T00:43:58.619 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-18T00:43:58.736 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-18T00:44:00.728 INFO:teuthology.orchestra.run.smithi042.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-18T00:44:00.745 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:44:00.800 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:00.904 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-18T00:44:00.904 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-18T00:44:00.905 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-18T00:44:01.313 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-18T00:44:01.362 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-18T00:44:01.777 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 113 MB in 5s (24.2 MB/s) 2024-09-18T00:44:01.973 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-18T00:44:02.968 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-18T00:44:02.990 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:44:03.084 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-18T00:44:03.584 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:44:03.629 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:04.024 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-18T00:44:04.028 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:44:04.129 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:06.322 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-18T00:44:06.343 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:44:06.433 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:06.950 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-18T00:44:06.971 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:44:07.008 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:10.441 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:44:10.564 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:44:10.914 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:44:11.043 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:44:11.388 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-18T00:44:11.411 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-18T00:44:11.446 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-18T00:44:14.093 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:44:14.219 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:44:14.586 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:44:14.714 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:44:15.042 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-18T00:44:15.065 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-18T00:44:15.100 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-18T00:44:17.374 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-18T00:44:17.411 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:44:17.448 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:20.079 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:44:20.201 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:44:20.544 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-18T00:44:20.673 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:20.799 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-18T00:44:20.911 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:20.920 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-18T00:44:20.957 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:44:21.012 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:22.238 INFO:teuthology.orchestra.run.smithi042.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:44:22.238 INFO:teuthology.orchestra.run.smithi042.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-18T00:44:22.238 INFO:teuthology.orchestra.run.smithi042.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:44:22.238 INFO:teuthology.orchestra.run.smithi042.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-18T00:44:22.413 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:24.198 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-18T00:44:24.324 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-18T00:44:24.450 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:25.340 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:44:25.456 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:44:25.807 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-18T00:44:25.843 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:25.893 INFO:teuthology.orchestra.run.smithi042.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-18T00:44:25.893 INFO:teuthology.orchestra.run.smithi042.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:44:25.936 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:26.079 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-18T00:44:26.208 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:27.531 INFO:teuthology.orchestra.run.smithi195.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:44:27.531 INFO:teuthology.orchestra.run.smithi195.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-18T00:44:27.531 INFO:teuthology.orchestra.run.smithi195.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:44:27.531 INFO:teuthology.orchestra.run.smithi195.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-18T00:44:27.685 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:29.069 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-18T00:44:29.218 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-18T00:44:29.655 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:31.515 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:44:31.574 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-18T00:44:31.574 INFO:teuthology.orchestra.run.smithi195.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:44:40.660 INFO:teuthology.orchestra.run.smithi042.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-18T00:44:40.660 INFO:teuthology.orchestra.run.smithi042.stdout:Sourcing file `/etc/default/grub' 2024-09-18T00:44:40.679 INFO:teuthology.orchestra.run.smithi042.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T00:44:40.698 INFO:teuthology.orchestra.run.smithi042.stdout:Generating grub configuration file ... 2024-09-18T00:44:40.972 INFO:teuthology.orchestra.run.smithi042.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:44:40.992 INFO:teuthology.orchestra.run.smithi042.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:44:41.281 INFO:teuthology.orchestra.run.smithi042.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:44:41.286 INFO:teuthology.orchestra.run.smithi042.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-18T00:44:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-18T00:44:41.355 INFO:teuthology.orchestra.run.smithi042.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-18T00:44:41.511 INFO:teuthology.orchestra.run.smithi042.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-18T00:44:41.511 INFO:teuthology.orchestra.run.smithi042.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-18T00:44:41.511 INFO:teuthology.orchestra.run.smithi042.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-18T00:44:41.551 INFO:teuthology.orchestra.run.smithi042.stdout:done 2024-09-18T00:44:42.085 DEBUG:teuthology.orchestra.run.smithi042:> dpkg -s linux-image-generic 2024-09-18T00:44:42.106 INFO:teuthology.orchestra.run.smithi042.stdout:Package: linux-image-generic 2024-09-18T00:44:42.106 INFO:teuthology.orchestra.run.smithi042.stdout:Status: install ok installed 2024-09-18T00:44:42.107 INFO:teuthology.orchestra.run.smithi042.stdout:Priority: optional 2024-09-18T00:44:42.107 INFO:teuthology.orchestra.run.smithi042.stdout:Section: kernel 2024-09-18T00:44:42.107 INFO:teuthology.orchestra.run.smithi042.stdout:Installed-Size: 21 2024-09-18T00:44:42.107 INFO:teuthology.orchestra.run.smithi042.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T00:44:42.107 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: amd64 2024-09-18T00:44:42.107 INFO:teuthology.orchestra.run.smithi042.stdout:Source: linux-meta 2024-09-18T00:44:42.107 INFO:teuthology.orchestra.run.smithi042.stdout:Version: 5.15.0.122.122 2024-09-18T00:44:42.107 INFO:teuthology.orchestra.run.smithi042.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-18T00:44:42.107 INFO:teuthology.orchestra.run.smithi042.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T00:44:42.107 INFO:teuthology.orchestra.run.smithi042.stdout:Recommends: thermald 2024-09-18T00:44:42.108 INFO:teuthology.orchestra.run.smithi042.stdout:Description: Generic Linux kernel image 2024-09-18T00:44:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: This package will always depend on the latest generic kernel image 2024-09-18T00:44:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: available. 2024-09-18T00:44:42.108 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-18T00:44:42.108 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T00:44:42.108 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T00:44:42.109 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-18T00:44:42.109 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi042.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T00:44:42.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-get clean 2024-09-18T00:44:42.260 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-get update 2024-09-18T00:44:42.381 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:44:42.391 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-18T00:44:42.408 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-18T00:44:42.501 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-18T00:44:43.812 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-18T00:44:43.831 DEBUG:teuthology.orchestra.run.smithi042:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T00:44:43.894 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-18T00:44:44.109 INFO:teuthology.orchestra.run.smithi042.stdout:Building dependency tree... 2024-09-18T00:44:44.111 INFO:teuthology.orchestra.run.smithi042.stdout:Reading state information... 2024-09-18T00:44:44.312 INFO:teuthology.orchestra.run.smithi042.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-18T00:44:44.312 INFO:teuthology.orchestra.run.smithi042.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:44:44.312 INFO:teuthology.orchestra.run.smithi042.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:44:44.312 INFO:teuthology.orchestra.run.smithi042.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:44:44.312 INFO:teuthology.orchestra.run.smithi042.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:44:44.313 INFO:teuthology.orchestra.run.smithi042.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-18T00:44:44.313 INFO:teuthology.orchestra.run.smithi042.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-18T00:44:44.313 INFO:teuthology.orchestra.run.smithi042.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:44:44.358 INFO:teuthology.orchestra.run.smithi042.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-18T00:44:44.361 DEBUG:teuthology.orchestra.run.smithi042:> dpkg -s linux-image-generic 2024-09-18T00:44:44.377 INFO:teuthology.orchestra.run.smithi042.stdout:Package: linux-image-generic 2024-09-18T00:44:44.377 INFO:teuthology.orchestra.run.smithi042.stdout:Status: install ok installed 2024-09-18T00:44:44.377 INFO:teuthology.orchestra.run.smithi042.stdout:Priority: optional 2024-09-18T00:44:44.377 INFO:teuthology.orchestra.run.smithi042.stdout:Section: kernel 2024-09-18T00:44:44.377 INFO:teuthology.orchestra.run.smithi042.stdout:Installed-Size: 21 2024-09-18T00:44:44.377 INFO:teuthology.orchestra.run.smithi042.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T00:44:44.377 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: amd64 2024-09-18T00:44:44.377 INFO:teuthology.orchestra.run.smithi042.stdout:Source: linux-meta 2024-09-18T00:44:44.378 INFO:teuthology.orchestra.run.smithi042.stdout:Version: 5.15.0.122.122 2024-09-18T00:44:44.378 INFO:teuthology.orchestra.run.smithi042.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-18T00:44:44.378 INFO:teuthology.orchestra.run.smithi042.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T00:44:44.378 INFO:teuthology.orchestra.run.smithi042.stdout:Recommends: thermald 2024-09-18T00:44:44.378 INFO:teuthology.orchestra.run.smithi042.stdout:Description: Generic Linux kernel image 2024-09-18T00:44:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: This package will always depend on the latest generic kernel image 2024-09-18T00:44:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: available. 2024-09-18T00:44:44.378 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-18T00:44:44.423 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.g7rUGLNy6e 2024-09-18T00:44:44.423 DEBUG:teuthology.orchestra.run.smithi042:> sudo cp /boot/grub/grub.cfg /tmp/tmp.g7rUGLNy6e 2024-09-18T00:44:44.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.g7rUGLNy6e 2024-09-18T00:44:44.600 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.g7rUGLNy6e is 10KB 2024-09-18T00:44:44.614 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.g7rUGLNy6e 2024-09-18T00:44:44.619 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-18T00:44:44.674 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:44:44.675 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-18T00:44:44.675 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-18T00:44:44.759 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-18T00:44:44.759 DEBUG:teuthology.orchestra.run.smithi042:> sudo update-grub 2024-09-18T00:44:45.595 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-18T00:44:45.595 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub' 2024-09-18T00:44:45.622 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T00:44:45.644 INFO:teuthology.orchestra.run.smithi195.stdout:Generating grub configuration file ... 2024-09-18T00:44:45.997 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:44:46.020 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:44:46.031 INFO:teuthology.orchestra.run.smithi042.stderr:Sourcing file `/etc/default/grub' 2024-09-18T00:44:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T00:44:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-09-18T00:44:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:44:46.223 INFO:teuthology.orchestra.run.smithi042.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:44:46.322 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:44:46.328 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-18T00:44:46.394 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-18T00:44:46.399 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-18T00:44:46.444 INFO:teuthology.orchestra.run.smithi042.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:44:46.450 INFO:teuthology.orchestra.run.smithi042.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-18T00:44:46.493 INFO:teuthology.orchestra.run.smithi042.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-18T00:44:46.500 INFO:teuthology.orchestra.run.smithi042.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-18T00:44:46.564 INFO:teuthology.orchestra.run.smithi195.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-18T00:44:46.564 INFO:teuthology.orchestra.run.smithi195.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-18T00:44:46.564 INFO:teuthology.orchestra.run.smithi195.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-18T00:44:46.606 INFO:teuthology.orchestra.run.smithi195.stdout:done 2024-09-18T00:44:46.662 INFO:teuthology.orchestra.run.smithi042.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-18T00:44:46.662 INFO:teuthology.orchestra.run.smithi042.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-18T00:44:46.663 INFO:teuthology.orchestra.run.smithi042.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-18T00:44:46.674 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-09-18T00:44:46.691 DEBUG:teuthology.orchestra.run.smithi042:> sudo shutdown -r now 2024-09-18T00:44:47.129 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2024-09-18T00:44:47.151 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2024-09-18T00:44:47.151 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 21 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.15.0.122.122 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2024-09-18T00:44:47.152 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2024-09-18T00:44:47.153 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-18T00:44:47.153 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T00:44:47.153 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T00:44:47.153 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-18T00:44:47.153 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T00:44:47.153 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2024-09-18T00:44:47.346 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-09-18T00:44:47.486 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-18T00:44:47.588 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:44:47.688 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-18T00:44:47.751 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-18T00:44:48.884 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-18T00:44:48.903 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T00:44:48.969 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-18T00:44:49.180 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-09-18T00:44:49.181 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-09-18T00:44:49.371 INFO:teuthology.orchestra.run.smithi195.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-18T00:44:49.371 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:44:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:44:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:44:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:44:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-18T00:44:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-18T00:44:49.372 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:44:49.413 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-18T00:44:49.414 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2024-09-18T00:44:49.430 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2024-09-18T00:44:49.430 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2024-09-18T00:44:49.430 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2024-09-18T00:44:49.430 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2024-09-18T00:44:49.431 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 21 2024-09-18T00:44:49.431 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T00:44:49.431 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2024-09-18T00:44:49.431 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2024-09-18T00:44:49.431 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.15.0.122.122 2024-09-18T00:44:49.431 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-18T00:44:49.431 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T00:44:49.432 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2024-09-18T00:44:49.432 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2024-09-18T00:44:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2024-09-18T00:44:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2024-09-18T00:44:49.432 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-09-18T00:44:49.475 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.zKyxjY7VTC 2024-09-18T00:44:49.475 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub/grub.cfg /tmp/tmp.zKyxjY7VTC 2024-09-18T00:44:49.531 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.zKyxjY7VTC 2024-09-18T00:44:49.648 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.zKyxjY7VTC is 10KB 2024-09-18T00:44:49.661 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.zKyxjY7VTC 2024-09-18T00:44:49.666 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-18T00:44:49.719 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:44:49.719 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-18T00:44:49.719 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-18T00:44:49.797 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-18T00:44:49.797 DEBUG:teuthology.orchestra.run.smithi195:> sudo update-grub 2024-09-18T00:44:51.196 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub' 2024-09-18T00:44:51.197 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T00:44:51.201 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2024-09-18T00:44:51.381 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:44:51.386 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:44:51.607 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:44:51.612 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-18T00:44:51.651 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-18T00:44:51.657 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-18T00:44:51.779 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-18T00:44:51.779 INFO:teuthology.orchestra.run.smithi195.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-18T00:44:51.779 INFO:teuthology.orchestra.run.smithi195.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-18T00:44:51.791 INFO:teuthology.orchestra.run.smithi195.stderr:done 2024-09-18T00:44:51.795 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2024-09-18T00:45:16.718 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-18T00:45:16.719 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-18T00:45:16.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:45:21.801 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-18T00:45:21.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-18T00:45:21.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:45:35.165 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-09-18T00:45:40.285 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-09-18T00:45:44.170 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-18T00:45:44.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:45:49.290 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-18T00:45:49.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:46:44.228 DEBUG:teuthology.orchestra.remote:timed out 2024-09-18T00:46:49.297 DEBUG:teuthology.orchestra.remote:timed out 2024-09-18T00:46:56.231 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-18T00:46:56.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:46:56.447 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-09-18T00:46:57.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-18T00:46:57.013 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-18T00:46:57.014 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-09-18T00:46:57.064 INFO:teuthology.orchestra.run.smithi042.stdout:5.15.0-122-generic 2024-09-18T00:46:57.064 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-18T00:46:57.064 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T00:46:57.064 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-18T00:46:58.065 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-18T00:46:58.066 DEBUG:teuthology.orchestra.run.smithi042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T00:46:58.129 INFO:teuthology.orchestra.run.smithi042.stdout:ttyS1 2024-09-18T00:46:58.162 DEBUG:teuthology.parallel:result is None 2024-09-18T00:47:01.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-18T00:47:01.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:47:01.557 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-09-18T00:47:02.157 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-18T00:47:02.157 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-18T00:47:02.157 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-09-18T00:47:02.201 INFO:teuthology.orchestra.run.smithi195.stdout:5.15.0-122-generic 2024-09-18T00:47:02.202 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-18T00:47:02.202 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T00:47:02.202 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-18T00:47:03.202 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-18T00:47:03.203 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T00:47:03.266 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2024-09-18T00:47:03.291 DEBUG:teuthology.parallel:result is None 2024-09-18T00:47:03.291 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-18T00:47:03.300 INFO:teuthology.task.internal:Creating test directory... 2024-09-18T00:47:03.300 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T00:47:03.303 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T00:47:03.308 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-18T00:47:03.315 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-18T00:47:03.323 INFO:teuthology.task.internal:Creating archive directory... 2024-09-18T00:47:03.323 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T00:47:03.349 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T00:47:03.370 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-18T00:47:03.376 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-18T00:47:03.377 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T00:47:03.410 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T00:47:03.428 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:47:03.429 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:47:03.436 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:47:03.438 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:47:03.439 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-18T00:47:03.449 INFO:teuthology.task.internal:Configuring sudo... 2024-09-18T00:47:03.449 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T00:47:03.485 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T00:47:03.507 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-18T00:47:03.516 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-18T00:47:03.516 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T00:47:03.541 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T00:47:03.554 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T00:47:03.593 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T00:47:03.641 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:47:03.641 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T00:47:03.707 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T00:47:03.713 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T00:47:03.762 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:47:03.763 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T00:47:03.822 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-09-18T00:47:03.825 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-09-18T00:47:03.898 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-18T00:47:03.908 INFO:teuthology.task.internal:Starting timer... 2024-09-18T00:47:03.908 INFO:teuthology.run_tasks:Running task pcp... 2024-09-18T00:47:03.918 INFO:teuthology.run_tasks:Running task selinux... 2024-09-18T00:47:03.926 DEBUG:teuthology.task.selinux:Excluding smithi042: OS 'ubuntu' does not support SELinux 2024-09-18T00:47:03.927 DEBUG:teuthology.task.selinux:Excluding smithi195: OS 'ubuntu' does not support SELinux 2024-09-18T00:47:03.927 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-18T00:47:03.927 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-18T00:47:03.927 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-18T00:47:03.927 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-18T00:47:03.935 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-18T00:47:03.939 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-18T00:47:04.068 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-18T00:47:04.079 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-18T00:47:04.080 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-18T00:54:38.613 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-09-18T00:54:38.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-18T00:54:38.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:54:38.700 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-09-18T00:54:38.781 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-18T00:54:38.781 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-18T00:54:38.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:54:38.859 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-09-18T00:54:38.937 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-18T00:54:38.937 INFO:teuthology.run_tasks:Running task clock... 2024-09-18T00:54:38.946 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-18T00:54:38.946 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T00:54:38.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T00:54:38.949 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T00:54:38.949 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T00:54:38.977 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-18T00:54:38.978 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: Command line: ntpd -gq 2024-09-18T00:54:38.978 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: ---------------------------------------------------- 2024-09-18T00:54:38.978 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: ntp-4 is maintained by Network Time Foundation, 2024-09-18T00:54:38.978 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-18T00:54:38.978 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: corporation. Support and training for ntp-4 are 2024-09-18T00:54:38.978 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: available at https://www.nwtime.org/support 2024-09-18T00:54:38.978 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: ---------------------------------------------------- 2024-09-18T00:54:38.979 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: proto: precision = 0.068 usec (-24) 2024-09-18T00:54:38.979 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: basedate set to 2022-02-04 2024-09-18T00:54:38.979 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: gps base set to 2022-02-06 (week 2196) 2024-09-18T00:54:38.979 INFO:teuthology.orchestra.run.smithi042.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T00:54:38.980 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T00:54:38.980 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: restrict ::: KOD does nothing without LIMITED. 2024-09-18T00:54:38.980 INFO:teuthology.orchestra.run.smithi042.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-18T00:54:38.981 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: Listen and drop on 0 v6wildcard [::]:123 2024-09-18T00:54:38.981 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-18T00:54:38.981 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: Listen normally on 2 lo 127.0.0.1:123 2024-09-18T00:54:38.981 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: Listen normally on 3 ens1f0 172.21.15.42:123 2024-09-18T00:54:38.981 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: Listen normally on 4 lo [::1]:123 2024-09-18T00:54:38.981 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1440%4]:123 2024-09-18T00:54:38.981 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:38 ntpd[18349]: Listening on routing socket on fd #22 for interface updates 2024-09-18T00:54:39.008 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-18T00:54:39.008 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: Command line: ntpd -gq 2024-09-18T00:54:39.008 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: ---------------------------------------------------- 2024-09-18T00:54:39.008 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: ntp-4 is maintained by Network Time Foundation, 2024-09-18T00:54:39.008 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-18T00:54:39.009 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: corporation. Support and training for ntp-4 are 2024-09-18T00:54:39.009 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: available at https://www.nwtime.org/support 2024-09-18T00:54:39.009 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: ---------------------------------------------------- 2024-09-18T00:54:39.009 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: proto: precision = 0.045 usec (-24) 2024-09-18T00:54:39.009 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: basedate set to 2022-02-04 2024-09-18T00:54:39.009 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: gps base set to 2022-02-06 (week 2196) 2024-09-18T00:54:39.009 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T00:54:39.010 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: restrict ::: KOD does nothing without LIMITED. 2024-09-18T00:54:39.010 INFO:teuthology.orchestra.run.smithi195.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T00:54:39.010 INFO:teuthology.orchestra.run.smithi195.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-18T00:54:39.011 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: Listen and drop on 0 v6wildcard [::]:123 2024-09-18T00:54:39.011 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-18T00:54:39.011 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: Listen normally on 2 lo 127.0.0.1:123 2024-09-18T00:54:39.011 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: Listen normally on 3 enp3s0f1 172.21.15.195:123 2024-09-18T00:54:39.011 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: Listen normally on 4 lo [::1]:123 2024-09-18T00:54:39.011 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8081%5]:123 2024-09-18T00:54:39.011 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:39 ntpd[18465]: Listening on routing socket on fd #22 for interface updates 2024-09-18T00:54:39.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:39 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:39.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:39 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:39.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:39 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:39.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:39 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:39.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:39 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:39.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:39 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:40.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:40.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:40.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:40.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:40.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:40.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:40.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:40.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:40.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:40.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:40.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:40.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:40 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:40.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:40 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:40.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:40 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:40.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:40 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:40.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:40 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:40.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:40 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:40.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:40 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:41.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:41 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:41.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:41 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:41.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:41 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:41.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:41 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:41.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:41 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:41.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:41 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:42.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:42.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:42.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:42 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:42.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:42 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:42.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:42 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:42.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:42 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:42.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:42 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:42.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:42 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:42.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:42 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:42.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:42 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:42.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:42 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:43.008 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:43 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:43.008 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:43 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:43.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:43 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:43.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:43 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:43.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:43 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:43.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:43 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:43.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:43.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:43 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:44.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:44.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:44.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:44.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:44.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:44.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:44.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:44.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:44.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:44.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:44.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:44.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:44 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:44.978 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:44 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:44.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:44 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:44.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:44 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:44.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:44 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:44.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:44 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:44.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:44 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:45.008 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:45 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:45.008 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:45 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:45.008 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:45 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:45.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:45 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:45.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:45 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:45.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:45 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:45.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:45.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:45.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:45.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:45.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:45.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:45.979 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:45.980 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:45.980 INFO:teuthology.orchestra.run.smithi042.stdout:18 Sep 00:54:45 ntpd[18349]: ntpd: time slew -0.000473 s 2024-09-18T00:54:45.980 INFO:teuthology.orchestra.run.smithi042.stdout:ntpd: time slew -0.000473s 2024-09-18T00:54:45.981 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-18T00:54:45.981 INFO:teuthology.orchestra.run.smithi042.stderr:18 Sep 00:54:45 ntpd[18349]: can't open /var/log/ntpstats/loopstats.20240918: Permission denied 2024-09-18T00:54:46.008 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:46 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:46.008 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:46 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:46.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:46 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:46.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:46 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:46.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:46 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:54:46.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:46 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:54:46.009 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:46 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:54:46.009 INFO:teuthology.orchestra.run.smithi195.stdout:18 Sep 00:54:46 ntpd[18465]: ntpd: time slew -0.003770 s 2024-09-18T00:54:46.009 INFO:teuthology.orchestra.run.smithi195.stdout:ntpd: time slew -0.003770s 2024-09-18T00:54:46.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:46 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:54:46.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:46 ntpd[18465]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-18T00:54:46.010 INFO:teuthology.orchestra.run.smithi195.stderr:18 Sep 00:54:46 ntpd[18465]: can't open /var/log/ntpstats/loopstats.20240918: Permission denied 2024-09-18T00:54:46.038 INFO:teuthology.orchestra.run.smithi042.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T00:54:46.038 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================== 2024-09-18T00:54:46.038 INFO:teuthology.orchestra.run.smithi042.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:54:46.038 INFO:teuthology.orchestra.run.smithi042.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:54:46.039 INFO:teuthology.orchestra.run.smithi042.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:54:46.039 INFO:teuthology.orchestra.run.smithi042.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:54:46.060 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T00:54:46.060 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2024-09-18T00:54:46.060 INFO:teuthology.orchestra.run.smithi195.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:54:46.060 INFO:teuthology.orchestra.run.smithi195.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:54:46.060 INFO:teuthology.orchestra.run.smithi195.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:54:46.060 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:54:46.061 INFO:teuthology.run_tasks:Running task install... 2024-09-18T00:54:46.070 DEBUG:teuthology.task.install:project ceph 2024-09-18T00:54:46.070 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}} 2024-09-18T00:54:46.070 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'} 2024-09-18T00:54:46.070 INFO:teuthology.task.install:Using flavor: default 2024-09-18T00:54:46.076 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-18T00:54:46.076 INFO:teuthology.task.install:extra packages: [] 2024-09-18T00:54:46.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-key list | grep Ceph 2024-09-18T00:54:46.077 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-key list | grep Ceph 2024-09-18T00:54:46.137 INFO:teuthology.orchestra.run.smithi042.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-18T00:54:46.160 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-18T00:54:46.165 INFO:teuthology.orchestra.run.smithi042.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-18T00:54:46.166 INFO:teuthology.orchestra.run.smithi042.stdout:uid [ unknown] Ceph.com (release key) 2024-09-18T00:54:46.167 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-18T00:54:46.167 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:54:46.190 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-18T00:54:46.190 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph.com (release key) 2024-09-18T00:54:46.191 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-18T00:54:46.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:54:46.312 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default/ 2024-09-18T00:54:46.312 INFO:teuthology.task.install.deb:Package version is 19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:54:46.344 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default/ 2024-09-18T00:54:46.344 INFO:teuthology.task.install.deb:Package version is 19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:54:46.422 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:54:46.423 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-18T00:54:46.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo apt-get update 2024-09-18T00:54:46.459 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:54:46.459 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-18T00:54:46.473 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-09-18T00:54:46.630 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-18T00:54:46.642 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:54:46.642 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-18T00:54:46.651 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-18T00:54:46.666 INFO:teuthology.orchestra.run.smithi042.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy InRelease 2024-09-18T00:54:46.668 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-18T00:54:46.688 INFO:teuthology.orchestra.run.smithi042.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-18T00:54:46.695 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy InRelease 2024-09-18T00:54:46.710 INFO:teuthology.orchestra.run.smithi042.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-18T00:54:46.717 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-18T00:54:46.731 INFO:teuthology.orchestra.run.smithi042.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-18T00:54:46.738 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-18T00:54:46.740 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:54:46.760 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-18T00:54:46.761 INFO:teuthology.orchestra.run.smithi042.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-09-18T00:54:46.787 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-09-18T00:54:47.002 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 28.9 kB in 0s (83.3 kB/s) 2024-09-18T00:54:48.004 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-18T00:54:48.021 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-330-gc2ddf972-1jammy cephadm=19.1.1-330-gc2ddf972-1jammy ceph-mds=19.1.1-330-gc2ddf972-1jammy ceph-mgr=19.1.1-330-gc2ddf972-1jammy ceph-common=19.1.1-330-gc2ddf972-1jammy ceph-fuse=19.1.1-330-gc2ddf972-1jammy ceph-test=19.1.1-330-gc2ddf972-1jammy ceph-volume=19.1.1-330-gc2ddf972-1jammy radosgw=19.1.1-330-gc2ddf972-1jammy python3-rados=19.1.1-330-gc2ddf972-1jammy python3-rgw=19.1.1-330-gc2ddf972-1jammy python3-cephfs=19.1.1-330-gc2ddf972-1jammy python3-rbd=19.1.1-330-gc2ddf972-1jammy libcephfs2=19.1.1-330-gc2ddf972-1jammy libcephfs-dev=19.1.1-330-gc2ddf972-1jammy librados2=19.1.1-330-gc2ddf972-1jammy librbd1=19.1.1-330-gc2ddf972-1jammy rbd-fuse=19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:54:48.083 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-18T00:54:48.299 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-09-18T00:54:48.300 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-09-18T00:54:48.506 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:54:48.506 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:54:48.506 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:54:48.506 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:54:48.506 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-18T00:54:48.506 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-18T00:54:48.506 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:54:48.508 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-09-18T00:54:48.508 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-18T00:54:48.508 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-18T00:54:48.508 INFO:teuthology.orchestra.run.smithi195.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-18T00:54:48.508 INFO:teuthology.orchestra.run.smithi195.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-18T00:54:48.508 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-18T00:54:48.508 INFO:teuthology.orchestra.run.smithi195.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-18T00:54:48.508 INFO:teuthology.orchestra.run.smithi195.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-18T00:54:48.508 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-18T00:54:48.509 INFO:teuthology.orchestra.run.smithi195.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-18T00:54:48.510 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-18T00:54:48.510 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-18T00:54:48.510 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet zip 2024-09-18T00:54:48.510 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-09-18T00:54:48.510 INFO:teuthology.orchestra.run.smithi195.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-18T00:54:48.510 INFO:teuthology.orchestra.run.smithi195.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-18T00:54:48.510 INFO:teuthology.orchestra.run.smithi195.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-18T00:54:48.510 INFO:teuthology.orchestra.run.smithi195.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-18T00:54:48.510 INFO:teuthology.orchestra.run.smithi195.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-18T00:54:48.510 INFO:teuthology.orchestra.run.smithi195.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-18T00:54:48.511 INFO:teuthology.orchestra.run.smithi195.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-18T00:54:48.511 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2024-09-18T00:54:48.511 INFO:teuthology.orchestra.run.smithi195.stdout: btrfs-tools 2024-09-18T00:54:48.613 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-09-18T00:54:48.613 INFO:teuthology.orchestra.run.smithi195.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-18T00:54:48.613 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-18T00:54:48.613 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-18T00:54:48.613 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-18T00:54:48.613 INFO:teuthology.orchestra.run.smithi195.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-18T00:54:48.613 INFO:teuthology.orchestra.run.smithi195.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-18T00:54:48.614 INFO:teuthology.orchestra.run.smithi195.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-18T00:54:48.614 INFO:teuthology.orchestra.run.smithi195.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-18T00:54:48.615 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-18T00:54:48.616 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-18T00:54:48.616 INFO:teuthology.orchestra.run.smithi195.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-18T00:54:48.616 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-18T00:54:48.616 INFO:teuthology.orchestra.run.smithi195.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-18T00:54:48.616 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-18T00:54:48.616 INFO:teuthology.orchestra.run.smithi195.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-18T00:54:48.641 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-18T00:54:48.722 INFO:teuthology.orchestra.run.smithi042.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-18T00:54:48.756 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 118 newly installed, 0 to remove and 331 not upgraded. 2024-09-18T00:54:48.756 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 181 MB of archives. 2024-09-18T00:54:48.757 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 829 MB of additional disk space will be used. 2024-09-18T00:54:48.757 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-330-gc2ddf972-1jammy [3,591 kB] 2024-09-18T00:54:48.802 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-18T00:54:48.885 INFO:teuthology.orchestra.run.smithi042.stdout:Fetched 28.9 kB in 2s (12.8 kB/s) 2024-09-18T00:54:48.938 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-330-gc2ddf972-1jammy [3,242 kB] 2024-09-18T00:54:48.993 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-330-gc2ddf972-1jammy [719 kB] 2024-09-18T00:54:49.004 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-330-gc2ddf972-1jammy [356 kB] 2024-09-18T00:54:49.009 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-330-gc2ddf972-1jammy [32.8 kB] 2024-09-18T00:54:49.010 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-330-gc2ddf972-1jammy [180 kB] 2024-09-18T00:54:49.013 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-330-gc2ddf972-1jammy [67.8 kB] 2024-09-18T00:54:49.014 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-330-gc2ddf972-1jammy [333 kB] 2024-09-18T00:54:49.019 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-330-gc2ddf972-1jammy [6,875 kB] 2024-09-18T00:54:49.126 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-330-gc2ddf972-1jammy [112 kB] 2024-09-18T00:54:49.128 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-330-gc2ddf972-1jammy [472 kB] 2024-09-18T00:54:49.134 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-330-gc2ddf972-1jammy [25.9 MB] 2024-09-18T00:54:49.223 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-18T00:54:49.235 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-18T00:54:49.316 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-18T00:54:49.545 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-18T00:54:49.557 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-18T00:54:49.586 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-18T00:54:49.595 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-18T00:54:49.596 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-18T00:54:49.597 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-18T00:54:49.674 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-18T00:54:49.743 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-330-gc2ddf972-1jammy [5,145 kB] 2024-09-18T00:54:49.743 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-18T00:54:49.747 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-18T00:54:49.749 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-18T00:54:49.750 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-18T00:54:49.754 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-18T00:54:49.759 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-18T00:54:49.762 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-18T00:54:49.763 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-18T00:54:49.790 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-18T00:54:49.817 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-330-gc2ddf972-1jammy [245 kB] 2024-09-18T00:54:49.818 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-330-gc2ddf972-1jammy [124 kB] 2024-09-18T00:54:49.819 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-330-gc2ddf972-1jammy [1,555 kB] 2024-09-18T00:54:49.833 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-330-gc2ddf972-1jammy [6,143 kB] 2024-09-18T00:54:49.846 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-18T00:54:49.847 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-18T00:54:49.847 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-18T00:54:49.848 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-18T00:54:49.849 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-18T00:54:49.850 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-18T00:54:49.850 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-18T00:54:49.873 INFO:teuthology.orchestra.run.smithi195.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-18T00:54:49.874 INFO:teuthology.orchestra.run.smithi195.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-18T00:54:49.942 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-18T00:54:49.951 INFO:teuthology.orchestra.run.smithi195.stdout:Get:47 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-330-gc2ddf972-1jammy [22.8 MB] 2024-09-18T00:54:49.954 INFO:teuthology.orchestra.run.smithi195.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-18T00:54:49.954 INFO:teuthology.orchestra.run.smithi195.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-18T00:54:49.956 INFO:teuthology.orchestra.run.smithi195.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-18T00:54:49.957 INFO:teuthology.orchestra.run.smithi195.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-18T00:54:49.958 INFO:teuthology.orchestra.run.smithi195.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-18T00:54:49.959 DEBUG:teuthology.orchestra.run.smithi042:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-330-gc2ddf972-1jammy cephadm=19.1.1-330-gc2ddf972-1jammy ceph-mds=19.1.1-330-gc2ddf972-1jammy ceph-mgr=19.1.1-330-gc2ddf972-1jammy ceph-common=19.1.1-330-gc2ddf972-1jammy ceph-fuse=19.1.1-330-gc2ddf972-1jammy ceph-test=19.1.1-330-gc2ddf972-1jammy ceph-volume=19.1.1-330-gc2ddf972-1jammy radosgw=19.1.1-330-gc2ddf972-1jammy python3-rados=19.1.1-330-gc2ddf972-1jammy python3-rgw=19.1.1-330-gc2ddf972-1jammy python3-cephfs=19.1.1-330-gc2ddf972-1jammy python3-rbd=19.1.1-330-gc2ddf972-1jammy libcephfs2=19.1.1-330-gc2ddf972-1jammy libcephfs-dev=19.1.1-330-gc2ddf972-1jammy librados2=19.1.1-330-gc2ddf972-1jammy librbd1=19.1.1-330-gc2ddf972-1jammy rbd-fuse=19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:54:49.960 INFO:teuthology.orchestra.run.smithi195.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-18T00:54:49.962 INFO:teuthology.orchestra.run.smithi195.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-18T00:54:50.024 INFO:teuthology.orchestra.run.smithi042.stdout:Reading package lists... 2024-09-18T00:54:50.036 INFO:teuthology.orchestra.run.smithi195.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-18T00:54:50.046 INFO:teuthology.orchestra.run.smithi195.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-18T00:54:50.046 INFO:teuthology.orchestra.run.smithi195.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-18T00:54:50.118 INFO:teuthology.orchestra.run.smithi195.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-18T00:54:50.118 INFO:teuthology.orchestra.run.smithi195.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-18T00:54:50.120 INFO:teuthology.orchestra.run.smithi195.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-18T00:54:50.124 INFO:teuthology.orchestra.run.smithi195.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-18T00:54:50.126 INFO:teuthology.orchestra.run.smithi195.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-18T00:54:50.128 INFO:teuthology.orchestra.run.smithi195.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-18T00:54:50.130 INFO:teuthology.orchestra.run.smithi195.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-18T00:54:50.198 INFO:teuthology.orchestra.run.smithi195.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-18T00:54:50.200 INFO:teuthology.orchestra.run.smithi195.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-18T00:54:50.201 INFO:teuthology.orchestra.run.smithi195.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-18T00:54:50.217 INFO:teuthology.orchestra.run.smithi042.stdout:Building dependency tree... 2024-09-18T00:54:50.218 INFO:teuthology.orchestra.run.smithi042.stdout:Reading state information... 2024-09-18T00:54:50.280 INFO:teuthology.orchestra.run.smithi195.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-18T00:54:50.422 INFO:teuthology.orchestra.run.smithi042.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:54:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:54:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:54:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:54:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-18T00:54:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-18T00:54:50.422 INFO:teuthology.orchestra.run.smithi042.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:54:50.422 INFO:teuthology.orchestra.run.smithi042.stdout:The following additional packages will be installed: 2024-09-18T00:54:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-18T00:54:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-18T00:54:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-18T00:54:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-18T00:54:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-18T00:54:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-18T00:54:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-18T00:54:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-18T00:54:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-18T00:54:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet zip 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout:Suggested packages: 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout:Recommended packages: 2024-09-18T00:54:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: btrfs-tools 2024-09-18T00:54:50.438 INFO:teuthology.orchestra.run.smithi195.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-18T00:54:50.459 INFO:teuthology.orchestra.run.smithi195.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-18T00:54:50.459 INFO:teuthology.orchestra.run.smithi195.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-330-gc2ddf972-1jammy [14.1 kB] 2024-09-18T00:54:50.459 INFO:teuthology.orchestra.run.smithi195.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-330-gc2ddf972-1jammy [877 kB] 2024-09-18T00:54:50.461 INFO:teuthology.orchestra.run.smithi195.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-18T00:54:50.461 INFO:teuthology.orchestra.run.smithi195.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-18T00:54:50.464 INFO:teuthology.orchestra.run.smithi195.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-18T00:54:50.473 INFO:teuthology.orchestra.run.smithi195.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-330-gc2ddf972-1jammy [2,462 kB] 2024-09-18T00:54:50.501 INFO:teuthology.orchestra.run.smithi195.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-18T00:54:50.504 INFO:teuthology.orchestra.run.smithi195.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-18T00:54:50.504 INFO:teuthology.orchestra.run.smithi195.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-18T00:54:50.507 INFO:teuthology.orchestra.run.smithi195.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-330-gc2ddf972-1jammy [794 kB] 2024-09-18T00:54:50.517 INFO:teuthology.orchestra.run.smithi195.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-330-gc2ddf972-1jammy [152 kB] 2024-09-18T00:54:50.518 INFO:teuthology.orchestra.run.smithi195.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-330-gc2ddf972-1jammy [2,275 kB] 2024-09-18T00:54:50.521 INFO:teuthology.orchestra.run.smithi042.stdout:The following NEW packages will be installed: 2024-09-18T00:54:50.521 INFO:teuthology.orchestra.run.smithi042.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-18T00:54:50.521 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-18T00:54:50.523 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-18T00:54:50.523 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-18T00:54:50.523 INFO:teuthology.orchestra.run.smithi042.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-18T00:54:50.523 INFO:teuthology.orchestra.run.smithi042.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-18T00:54:50.523 INFO:teuthology.orchestra.run.smithi042.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-18T00:54:50.523 INFO:teuthology.orchestra.run.smithi042.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-18T00:54:50.524 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-18T00:54:50.525 INFO:teuthology.orchestra.run.smithi042.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-18T00:54:50.525 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-18T00:54:50.525 INFO:teuthology.orchestra.run.smithi042.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-18T00:54:50.525 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-18T00:54:50.525 INFO:teuthology.orchestra.run.smithi042.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-18T00:54:50.541 INFO:teuthology.orchestra.run.smithi195.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-330-gc2ddf972-1jammy [8,625 kB] 2024-09-18T00:54:50.558 INFO:teuthology.orchestra.run.smithi195.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-18T00:54:50.558 INFO:teuthology.orchestra.run.smithi195.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-18T00:54:50.605 INFO:teuthology.orchestra.run.smithi195.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-18T00:54:50.605 INFO:teuthology.orchestra.run.smithi195.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-18T00:54:50.606 INFO:teuthology.orchestra.run.smithi195.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-18T00:54:50.607 INFO:teuthology.orchestra.run.smithi195.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-18T00:54:50.608 INFO:teuthology.orchestra.run.smithi195.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-18T00:54:50.609 INFO:teuthology.orchestra.run.smithi195.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-18T00:54:50.616 INFO:teuthology.orchestra.run.smithi195.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-18T00:54:50.669 INFO:teuthology.orchestra.run.smithi042.stdout:0 upgraded, 118 newly installed, 0 to remove and 331 not upgraded. 2024-09-18T00:54:50.669 INFO:teuthology.orchestra.run.smithi042.stdout:Need to get 181 MB of archives. 2024-09-18T00:54:50.669 INFO:teuthology.orchestra.run.smithi042.stdout:After this operation, 829 MB of additional disk space will be used. 2024-09-18T00:54:50.669 INFO:teuthology.orchestra.run.smithi042.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-330-gc2ddf972-1jammy [3,591 kB] 2024-09-18T00:54:50.678 INFO:teuthology.orchestra.run.smithi195.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-330-gc2ddf972-1jammy [14.2 kB] 2024-09-18T00:54:50.679 INFO:teuthology.orchestra.run.smithi195.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-330-gc2ddf972-1jammy [51.4 MB] 2024-09-18T00:54:50.686 INFO:teuthology.orchestra.run.smithi195.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-18T00:54:50.689 INFO:teuthology.orchestra.run.smithi195.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-18T00:54:50.690 INFO:teuthology.orchestra.run.smithi195.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-18T00:54:50.712 INFO:teuthology.orchestra.run.smithi042.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-18T00:54:50.767 INFO:teuthology.orchestra.run.smithi195.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-18T00:54:50.774 INFO:teuthology.orchestra.run.smithi195.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-18T00:54:50.776 INFO:teuthology.orchestra.run.smithi195.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-18T00:54:50.777 INFO:teuthology.orchestra.run.smithi195.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-18T00:54:50.788 INFO:teuthology.orchestra.run.smithi195.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-18T00:54:50.790 INFO:teuthology.orchestra.run.smithi195.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-18T00:54:50.798 INFO:teuthology.orchestra.run.smithi195.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-18T00:54:50.849 INFO:teuthology.orchestra.run.smithi195.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-18T00:54:50.850 INFO:teuthology.orchestra.run.smithi195.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-18T00:54:50.851 INFO:teuthology.orchestra.run.smithi195.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-18T00:54:50.919 INFO:teuthology.orchestra.run.smithi042.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-330-gc2ddf972-1jammy [3,242 kB] 2024-09-18T00:54:50.930 INFO:teuthology.orchestra.run.smithi195.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-18T00:54:50.932 INFO:teuthology.orchestra.run.smithi195.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-18T00:54:50.949 INFO:teuthology.orchestra.run.smithi195.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-18T00:54:50.950 INFO:teuthology.orchestra.run.smithi195.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-18T00:54:50.951 INFO:teuthology.orchestra.run.smithi195.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-18T00:54:50.955 INFO:teuthology.orchestra.run.smithi195.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-18T00:54:50.957 INFO:teuthology.orchestra.run.smithi195.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-18T00:54:51.042 INFO:teuthology.orchestra.run.smithi042.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-330-gc2ddf972-1jammy [719 kB] 2024-09-18T00:54:51.057 INFO:teuthology.orchestra.run.smithi042.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-330-gc2ddf972-1jammy [356 kB] 2024-09-18T00:54:51.082 INFO:teuthology.orchestra.run.smithi042.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-330-gc2ddf972-1jammy [32.8 kB] 2024-09-18T00:54:51.083 INFO:teuthology.orchestra.run.smithi042.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-330-gc2ddf972-1jammy [180 kB] 2024-09-18T00:54:51.087 INFO:teuthology.orchestra.run.smithi042.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-330-gc2ddf972-1jammy [67.8 kB] 2024-09-18T00:54:51.089 INFO:teuthology.orchestra.run.smithi042.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-330-gc2ddf972-1jammy [333 kB] 2024-09-18T00:54:51.096 INFO:teuthology.orchestra.run.smithi042.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-330-gc2ddf972-1jammy [6,875 kB] 2024-09-18T00:54:51.133 INFO:teuthology.orchestra.run.smithi042.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-18T00:54:51.145 INFO:teuthology.orchestra.run.smithi042.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-18T00:54:51.227 INFO:teuthology.orchestra.run.smithi042.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-18T00:54:51.324 INFO:teuthology.orchestra.run.smithi042.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-330-gc2ddf972-1jammy [112 kB] 2024-09-18T00:54:51.345 INFO:teuthology.orchestra.run.smithi042.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-330-gc2ddf972-1jammy [472 kB] 2024-09-18T00:54:51.355 INFO:teuthology.orchestra.run.smithi042.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-330-gc2ddf972-1jammy [25.9 MB] 2024-09-18T00:54:51.457 INFO:teuthology.orchestra.run.smithi042.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-18T00:54:51.467 INFO:teuthology.orchestra.run.smithi042.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-18T00:54:51.496 INFO:teuthology.orchestra.run.smithi042.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-18T00:54:51.505 INFO:teuthology.orchestra.run.smithi042.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-18T00:54:51.506 INFO:teuthology.orchestra.run.smithi042.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-18T00:54:51.507 INFO:teuthology.orchestra.run.smithi042.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-18T00:54:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-18T00:54:51.530 INFO:teuthology.orchestra.run.smithi042.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-18T00:54:51.534 INFO:teuthology.orchestra.run.smithi042.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-18T00:54:51.538 INFO:teuthology.orchestra.run.smithi042.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-18T00:54:51.619 INFO:teuthology.orchestra.run.smithi042.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-18T00:54:51.623 INFO:teuthology.orchestra.run.smithi042.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-18T00:54:51.627 INFO:teuthology.orchestra.run.smithi042.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-18T00:54:51.631 INFO:teuthology.orchestra.run.smithi042.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-18T00:54:51.633 INFO:teuthology.orchestra.run.smithi042.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-18T00:54:51.633 INFO:teuthology.orchestra.run.smithi042.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-18T00:54:51.635 INFO:teuthology.orchestra.run.smithi042.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-18T00:54:51.636 INFO:teuthology.orchestra.run.smithi042.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-18T00:54:51.637 INFO:teuthology.orchestra.run.smithi042.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-18T00:54:51.701 INFO:teuthology.orchestra.run.smithi042.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-18T00:54:51.720 INFO:teuthology.orchestra.run.smithi042.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-18T00:54:51.720 INFO:teuthology.orchestra.run.smithi042.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-18T00:54:51.721 INFO:teuthology.orchestra.run.smithi042.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-18T00:54:51.726 INFO:teuthology.orchestra.run.smithi042.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-18T00:54:51.728 INFO:teuthology.orchestra.run.smithi042.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-18T00:54:51.783 INFO:teuthology.orchestra.run.smithi042.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-18T00:54:51.783 INFO:teuthology.orchestra.run.smithi042.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-18T00:54:51.785 INFO:teuthology.orchestra.run.smithi042.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-18T00:54:51.786 INFO:teuthology.orchestra.run.smithi042.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-18T00:54:51.865 INFO:teuthology.orchestra.run.smithi042.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-18T00:54:51.866 INFO:teuthology.orchestra.run.smithi042.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-18T00:54:51.868 INFO:teuthology.orchestra.run.smithi042.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-18T00:54:51.869 INFO:teuthology.orchestra.run.smithi042.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-18T00:54:51.880 INFO:teuthology.orchestra.run.smithi042.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-18T00:54:51.945 INFO:teuthology.orchestra.run.smithi042.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-18T00:54:51.946 INFO:teuthology.orchestra.run.smithi042.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-18T00:54:51.947 INFO:teuthology.orchestra.run.smithi042.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-18T00:54:51.950 INFO:teuthology.orchestra.run.smithi042.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-18T00:54:51.954 INFO:teuthology.orchestra.run.smithi042.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-18T00:54:52.027 INFO:teuthology.orchestra.run.smithi042.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-18T00:54:52.029 INFO:teuthology.orchestra.run.smithi042.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-18T00:54:52.031 INFO:teuthology.orchestra.run.smithi042.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-18T00:54:52.035 INFO:teuthology.orchestra.run.smithi042.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-18T00:54:52.036 INFO:teuthology.orchestra.run.smithi042.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-18T00:54:52.109 INFO:teuthology.orchestra.run.smithi042.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-18T00:54:52.156 INFO:teuthology.orchestra.run.smithi042.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-18T00:54:52.224 INFO:teuthology.orchestra.run.smithi042.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-330-gc2ddf972-1jammy [5,145 kB] 2024-09-18T00:54:52.293 INFO:teuthology.orchestra.run.smithi042.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-18T00:54:52.295 INFO:teuthology.orchestra.run.smithi042.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-18T00:54:52.297 INFO:teuthology.orchestra.run.smithi042.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-18T00:54:52.297 INFO:teuthology.orchestra.run.smithi042.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-18T00:54:52.300 INFO:teuthology.orchestra.run.smithi042.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-330-gc2ddf972-1jammy [245 kB] 2024-09-18T00:54:52.304 INFO:teuthology.orchestra.run.smithi042.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-330-gc2ddf972-1jammy [124 kB] 2024-09-18T00:54:52.307 INFO:teuthology.orchestra.run.smithi042.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-330-gc2ddf972-1jammy [1,555 kB] 2024-09-18T00:54:52.307 INFO:teuthology.orchestra.run.smithi042.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-18T00:54:52.349 INFO:teuthology.orchestra.run.smithi042.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-330-gc2ddf972-1jammy [6,143 kB] 2024-09-18T00:54:52.367 INFO:teuthology.orchestra.run.smithi042.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-18T00:54:52.371 INFO:teuthology.orchestra.run.smithi042.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-18T00:54:52.371 INFO:teuthology.orchestra.run.smithi042.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-18T00:54:52.439 INFO:teuthology.orchestra.run.smithi042.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-18T00:54:52.439 INFO:teuthology.orchestra.run.smithi042.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-18T00:54:52.459 INFO:teuthology.orchestra.run.smithi042.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-18T00:54:52.460 INFO:teuthology.orchestra.run.smithi042.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-18T00:54:52.461 INFO:teuthology.orchestra.run.smithi042.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-18T00:54:52.462 INFO:teuthology.orchestra.run.smithi042.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-18T00:54:52.463 INFO:teuthology.orchestra.run.smithi042.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-18T00:54:52.464 INFO:teuthology.orchestra.run.smithi042.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-18T00:54:52.470 INFO:teuthology.orchestra.run.smithi042.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-18T00:54:52.478 INFO:teuthology.orchestra.run.smithi042.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-330-gc2ddf972-1jammy [22.8 MB] 2024-09-18T00:54:52.518 INFO:teuthology.orchestra.run.smithi195.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-330-gc2ddf972-1jammy [130 kB] 2024-09-18T00:54:52.521 INFO:teuthology.orchestra.run.smithi195.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-330-gc2ddf972-1jammy [39.0 kB] 2024-09-18T00:54:52.522 INFO:teuthology.orchestra.run.smithi195.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-330-gc2ddf972-1jammy [13.6 MB] 2024-09-18T00:54:52.541 INFO:teuthology.orchestra.run.smithi042.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-18T00:54:52.544 INFO:teuthology.orchestra.run.smithi042.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-18T00:54:52.546 INFO:teuthology.orchestra.run.smithi042.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-18T00:54:52.623 INFO:teuthology.orchestra.run.smithi042.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-18T00:54:52.629 INFO:teuthology.orchestra.run.smithi042.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-18T00:54:52.632 INFO:teuthology.orchestra.run.smithi042.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-18T00:54:52.633 INFO:teuthology.orchestra.run.smithi042.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-18T00:54:52.644 INFO:teuthology.orchestra.run.smithi042.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-18T00:54:52.645 INFO:teuthology.orchestra.run.smithi042.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-18T00:54:52.653 INFO:teuthology.orchestra.run.smithi042.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-18T00:54:52.704 INFO:teuthology.orchestra.run.smithi042.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-18T00:54:52.706 INFO:teuthology.orchestra.run.smithi042.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-18T00:54:52.707 INFO:teuthology.orchestra.run.smithi042.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-18T00:54:52.786 INFO:teuthology.orchestra.run.smithi042.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-18T00:54:52.787 INFO:teuthology.orchestra.run.smithi042.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-18T00:54:52.805 INFO:teuthology.orchestra.run.smithi042.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-18T00:54:52.805 INFO:teuthology.orchestra.run.smithi042.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-18T00:54:52.806 INFO:teuthology.orchestra.run.smithi042.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-18T00:54:52.812 INFO:teuthology.orchestra.run.smithi042.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-18T00:54:52.813 INFO:teuthology.orchestra.run.smithi042.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-18T00:54:52.960 INFO:teuthology.orchestra.run.smithi195.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-330-gc2ddf972-1jammy [91.4 kB] 2024-09-18T00:54:53.108 INFO:teuthology.orchestra.run.smithi042.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-330-gc2ddf972-1jammy [14.1 kB] 2024-09-18T00:54:53.108 INFO:teuthology.orchestra.run.smithi042.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-330-gc2ddf972-1jammy [877 kB] 2024-09-18T00:54:53.115 INFO:teuthology.orchestra.run.smithi042.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-330-gc2ddf972-1jammy [2,462 kB] 2024-09-18T00:54:53.136 INFO:teuthology.orchestra.run.smithi042.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-330-gc2ddf972-1jammy [794 kB] 2024-09-18T00:54:53.142 INFO:teuthology.orchestra.run.smithi042.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-330-gc2ddf972-1jammy [152 kB] 2024-09-18T00:54:53.143 INFO:teuthology.orchestra.run.smithi042.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-330-gc2ddf972-1jammy [2,275 kB] 2024-09-18T00:54:53.166 INFO:teuthology.orchestra.run.smithi042.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-330-gc2ddf972-1jammy [8,625 kB] 2024-09-18T00:54:53.309 INFO:teuthology.orchestra.run.smithi042.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-330-gc2ddf972-1jammy [14.2 kB] 2024-09-18T00:54:53.309 INFO:teuthology.orchestra.run.smithi042.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-330-gc2ddf972-1jammy [51.4 MB] 2024-09-18T00:54:53.532 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 181 MB in 4s (41.7 MB/s) 2024-09-18T00:54:53.670 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-18T00:54:53.770 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-18T00:54:53.776 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-18T00:54:53.844 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-18T00:54:54.189 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-18T00:54:54.216 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-18T00:54:54.247 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-18T00:54:54.418 INFO:teuthology.orchestra.run.smithi042.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-330-gc2ddf972-1jammy [130 kB] 2024-09-18T00:54:54.420 INFO:teuthology.orchestra.run.smithi042.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-330-gc2ddf972-1jammy [39.0 kB] 2024-09-18T00:54:54.421 INFO:teuthology.orchestra.run.smithi042.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-330-gc2ddf972-1jammy [13.6 MB] 2024-09-18T00:54:54.550 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-18T00:54:54.577 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-18T00:54:54.612 INFO:teuthology.orchestra.run.smithi042.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-330-gc2ddf972-1jammy [91.4 kB] 2024-09-18T00:54:54.633 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-18T00:54:55.070 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-18T00:54:55.097 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:54:55.146 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:54:55.208 INFO:teuthology.orchestra.run.smithi042.stdout:Fetched 181 MB in 4s (44.4 MB/s) 2024-09-18T00:54:55.377 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-18T00:54:55.479 INFO:teuthology.orchestra.run.smithi042.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-18T00:54:55.484 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-18T00:54:55.507 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-18T00:54:55.533 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:54:55.565 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:54:55.567 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-18T00:54:55.902 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-18T00:54:55.913 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-18T00:54:55.928 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:54:55.939 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-18T00:54:55.968 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:54:55.979 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-18T00:54:56.283 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-18T00:54:56.310 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-18T00:54:56.313 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-18T00:54:56.339 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-18T00:54:56.349 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-18T00:54:56.379 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-18T00:54:56.666 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librados2. 2024-09-18T00:54:56.692 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../007-librados2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:54:56.732 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librados2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:54:56.795 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-18T00:54:56.822 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:54:56.870 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:54:57.189 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librbd1. 2024-09-18T00:54:57.211 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../008-librbd1_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:54:57.251 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librbd1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:54:57.373 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-18T00:54:57.400 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:54:57.440 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:54:57.688 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs2. 2024-09-18T00:54:57.714 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:54:57.754 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:54:57.785 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-18T00:54:57.812 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:54:57.851 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:54:58.057 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rados. 2024-09-18T00:54:58.083 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../010-python3-rados_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:54:58.132 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rados (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:54:58.204 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-18T00:54:58.231 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-18T00:54:58.271 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-18T00:54:58.401 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-18T00:54:58.428 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:54:58.468 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-argparse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:54:58.566 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package librados2. 2024-09-18T00:54:58.593 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../007-librados2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:54:58.632 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librados2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:54:58.703 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cephfs. 2024-09-18T00:54:58.719 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:54:58.762 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cephfs (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:54:59.069 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package librbd1. 2024-09-18T00:54:59.073 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-18T00:54:59.095 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../008-librbd1_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:54:59.099 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:54:59.135 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librbd1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:54:59.139 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:54:59.451 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-18T00:54:59.478 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-18T00:54:59.518 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-18T00:54:59.571 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libcephfs2. 2024-09-18T00:54:59.599 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:54:59.638 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libcephfs2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:54:59.795 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-prettytable. 2024-09-18T00:54:59.822 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-18T00:54:59.853 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-18T00:54:59.924 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rados. 2024-09-18T00:54:59.940 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../010-python3-rados_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:54:59.982 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rados (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:00.081 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rbd. 2024-09-18T00:55:00.096 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:00.139 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rbd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:00.252 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-18T00:55:00.268 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:00.302 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-ceph-argparse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:00.493 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-18T00:55:00.520 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-18T00:55:00.559 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-18T00:55:00.563 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cephfs. 2024-09-18T00:55:00.580 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:00.621 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cephfs (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:00.828 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-18T00:55:00.843 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-18T00:55:00.887 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-18T00:55:00.941 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-18T00:55:00.968 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:01.189 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-18T00:55:01.205 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-18T00:55:01.239 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-18T00:55:01.310 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-18T00:55:01.326 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-18T00:55:01.386 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-18T00:55:01.601 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua5.1. 2024-09-18T00:55:01.627 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-18T00:55:01.693 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-18T00:55:01.914 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-prettytable. 2024-09-18T00:55:01.931 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-18T00:55:01.972 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-18T00:55:01.996 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-any. 2024-09-18T00:55:02.022 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-18T00:55:02.062 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-18T00:55:02.233 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rbd. 2024-09-18T00:55:02.260 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:02.298 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package zip. 2024-09-18T00:55:02.300 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rbd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:02.314 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-18T00:55:02.348 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking zip (3.0-12build2) ... 2024-09-18T00:55:02.660 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package unzip. 2024-09-18T00:55:02.686 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-18T00:55:02.726 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-18T00:55:02.736 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-18T00:55:02.763 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-18T00:55:02.803 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-18T00:55:03.064 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-18T00:55:03.080 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-18T00:55:03.080 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package luarocks. 2024-09-18T00:55:03.107 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-18T00:55:03.122 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-18T00:55:03.138 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-18T00:55:03.417 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-18T00:55:03.444 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-18T00:55:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-18T00:55:03.515 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librgw2. 2024-09-18T00:55:03.541 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../025-librgw2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:03.581 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librgw2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:03.804 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package lua5.1. 2024-09-18T00:55:03.830 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-18T00:55:03.870 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-18T00:55:04.109 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rgw. 2024-09-18T00:55:04.124 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:04.157 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package lua-any. 2024-09-18T00:55:04.158 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:04.172 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-18T00:55:04.207 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-18T00:55:04.443 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package zip. 2024-09-18T00:55:04.459 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-18T00:55:04.478 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-18T00:55:04.501 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking zip (3.0-12build2) ... 2024-09-18T00:55:04.505 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-18T00:55:04.545 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-18T00:55:04.821 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package unzip. 2024-09-18T00:55:04.848 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-18T00:55:04.881 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libradosstriper1. 2024-09-18T00:55:04.888 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-18T00:55:04.908 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:04.948 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libradosstriper1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package luarocks. 2024-09-18T00:55:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-18T00:55:05.318 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-18T00:55:05.326 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-common. 2024-09-18T00:55:05.353 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../029-ceph-common_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:05.393 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:05.751 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package librgw2. 2024-09-18T00:55:05.779 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../025-librgw2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:05.818 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking librgw2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:06.387 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rgw. 2024-09-18T00:55:06.403 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:06.445 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:06.748 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-18T00:55:06.764 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-base. 2024-09-18T00:55:06.775 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-18T00:55:06.793 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../030-ceph-base_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:06.814 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-18T00:55:06.841 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-base (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:07.151 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libradosstriper1. 2024-09-18T00:55:07.179 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:07.218 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libradosstriper1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:07.655 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-common. 2024-09-18T00:55:07.682 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../029-ceph-common_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:07.721 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:07.735 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-18T00:55:07.762 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-18T00:55:07.802 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-18T00:55:08.130 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cheroot. 2024-09-18T00:55:08.156 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-18T00:55:08.196 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-18T00:55:08.491 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-18T00:55:08.506 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-18T00:55:08.541 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-18T00:55:08.819 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-18T00:55:08.845 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-18T00:55:08.885 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-18T00:55:09.144 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-base. 2024-09-18T00:55:09.170 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../030-ceph-base_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:09.188 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-18T00:55:09.215 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-18T00:55:09.219 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-base (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:09.247 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-18T00:55:09.549 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempora. 2024-09-18T00:55:09.576 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-18T00:55:09.616 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-18T00:55:09.772 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-18T00:55:09.800 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-18T00:55:09.839 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-18T00:55:09.944 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-portend. 2024-09-18T00:55:09.970 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-18T00:55:10.010 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-18T00:55:10.117 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cheroot. 2024-09-18T00:55:10.144 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-18T00:55:10.183 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-18T00:55:10.288 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-18T00:55:10.315 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-18T00:55:10.355 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-18T00:55:10.503 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-18T00:55:10.530 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-18T00:55:10.569 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-18T00:55:10.607 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-18T00:55:10.635 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-18T00:55:10.674 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-18T00:55:10.856 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-18T00:55:10.883 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-18T00:55:10.922 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-18T00:55:11.003 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-natsort. 2024-09-18T00:55:11.029 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-18T00:55:11.061 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-18T00:55:11.200 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-18T00:55:11.227 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-18T00:55:11.258 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-18T00:55:11.389 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-logutils. 2024-09-18T00:55:11.415 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-18T00:55:11.455 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-18T00:55:11.528 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-tempora. 2024-09-18T00:55:11.555 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-18T00:55:11.594 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-18T00:55:11.758 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-18T00:55:11.785 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-18T00:55:11.825 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-18T00:55:11.897 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-portend. 2024-09-18T00:55:11.924 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-18T00:55:11.964 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-18T00:55:12.111 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-mako. 2024-09-18T00:55:12.138 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-18T00:55:12.178 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-18T00:55:12.483 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-18T00:55:12.497 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-18T00:55:12.510 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-18T00:55:12.524 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-18T00:55:12.550 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-18T00:55:12.589 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-18T00:55:12.811 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-18T00:55:12.838 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-18T00:55:12.875 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-18T00:55:12.902 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-18T00:55:12.928 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-18T00:55:12.942 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-18T00:55:13.203 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webob. 2024-09-18T00:55:13.218 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-18T00:55:13.253 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-18T00:55:13.273 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-natsort. 2024-09-18T00:55:13.301 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-18T00:55:13.339 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-18T00:55:13.556 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-waitress. 2024-09-18T00:55:13.582 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-18T00:55:13.626 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-logutils. 2024-09-18T00:55:13.642 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-18T00:55:13.684 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-18T00:55:13.708 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-18T00:55:13.979 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-18T00:55:14.006 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-18T00:55:14.017 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempita. 2024-09-18T00:55:14.044 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-18T00:55:14.045 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-18T00:55:14.084 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-18T00:55:14.348 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-mako. 2024-09-18T00:55:14.375 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-18T00:55:14.386 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-paste. 2024-09-18T00:55:14.413 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-18T00:55:14.414 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-18T00:55:14.453 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-18T00:55:14.743 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-18T00:55:14.770 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-18T00:55:14.798 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-18T00:55:14.809 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-18T00:55:14.824 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-18T00:55:14.864 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-18T00:55:15.112 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-18T00:55:15.117 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-18T00:55:15.140 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-18T00:55:15.144 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-18T00:55:15.179 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-18T00:55:15.184 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-18T00:55:15.457 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-webob. 2024-09-18T00:55:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-18T00:55:15.487 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webtest. 2024-09-18T00:55:15.513 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-18T00:55:15.514 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-18T00:55:15.553 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-18T00:55:15.818 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-waitress. 2024-09-18T00:55:15.845 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-18T00:55:15.864 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pecan. 2024-09-18T00:55:15.879 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-18T00:55:15.891 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-18T00:55:15.931 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-18T00:55:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-tempita. 2024-09-18T00:55:16.198 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-18T00:55:16.237 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-18T00:55:16.278 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-certifi. 2024-09-18T00:55:16.305 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-18T00:55:16.344 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-18T00:55:16.523 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-paste. 2024-09-18T00:55:16.551 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-18T00:55:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-18T00:55:16.664 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-idna. 2024-09-18T00:55:16.690 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-18T00:55:16.730 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-18T00:55:16.893 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-18T00:55:16.909 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-18T00:55:16.951 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-18T00:55:17.042 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-urllib3. 2024-09-18T00:55:17.068 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-18T00:55:17.108 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-18T00:55:17.254 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-18T00:55:17.282 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-18T00:55:17.321 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-18T00:55:17.445 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests. 2024-09-18T00:55:17.472 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-18T00:55:17.511 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-18T00:55:17.607 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-webtest. 2024-09-18T00:55:17.634 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-18T00:55:17.674 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-18T00:55:17.977 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pecan. 2024-09-18T00:55:18.005 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-18T00:55:18.046 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-18T00:55:18.716 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-certifi. 2024-09-18T00:55:18.717 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-18T00:55:18.717 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-18T00:55:18.717 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-18T00:55:18.717 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-18T00:55:18.717 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-18T00:55:18.718 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-18T00:55:18.718 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:18.718 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-modules-core (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:18.768 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-idna. 2024-09-18T00:55:18.795 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-18T00:55:18.843 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-18T00:55:18.865 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-18T00:55:18.891 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:18.930 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:19.170 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-urllib3. 2024-09-18T00:55:19.198 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-18T00:55:19.237 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-18T00:55:19.241 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-18T00:55:19.268 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-18T00:55:19.307 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-18T00:55:19.565 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-requests. 2024-09-18T00:55:19.593 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-18T00:55:19.602 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-openssl. 2024-09-18T00:55:19.629 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-18T00:55:19.631 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-18T00:55:19.669 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-18T00:55:19.951 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-18T00:55:19.979 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-18T00:55:20.005 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr. 2024-09-18T00:55:20.018 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-18T00:55:20.032 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:20.072 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:20.337 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-18T00:55:20.354 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-modules-core (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:20.425 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mon. 2024-09-18T00:55:20.440 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:20.474 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mon (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:20.800 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-18T00:55:20.828 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:20.867 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:21.045 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-osd. 2024-09-18T00:55:21.071 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:21.112 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-osd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:21.178 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-18T00:55:21.205 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-18T00:55:21.245 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-18T00:55:21.556 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-openssl. 2024-09-18T00:55:21.583 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-18T00:55:21.622 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-18T00:55:22.860 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr. 2024-09-18T00:55:22.887 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:22.926 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:22.948 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph. 2024-09-18T00:55:22.975 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../066-ceph_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:23.015 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:23.329 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mon. 2024-09-18T00:55:23.342 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-fuse. 2024-09-18T00:55:23.356 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:23.369 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:23.395 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mon (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:23.409 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:23.746 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mds. 2024-09-18T00:55:23.772 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:23.812 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mds (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:24.007 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-osd. 2024-09-18T00:55:24.034 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:24.074 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-osd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:24.215 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephadm. 2024-09-18T00:55:24.241 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../069-cephadm_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:24.281 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:24.609 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-18T00:55:24.636 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-18T00:55:24.675 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-18T00:55:24.978 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-18T00:55:24.994 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-18T00:55:25.037 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-18T00:55:25.135 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph. 2024-09-18T00:55:25.152 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../066-ceph_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:25.185 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:25.472 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-fuse. 2024-09-18T00:55:25.498 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:25.538 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:25.908 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mds. 2024-09-18T00:55:25.935 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:25.974 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mds (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:26.000 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-babel. 2024-09-18T00:55:26.027 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-18T00:55:26.066 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-18T00:55:26.336 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jinja2. 2024-09-18T00:55:26.362 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-18T00:55:26.386 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package cephadm. 2024-09-18T00:55:26.394 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-18T00:55:26.413 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../069-cephadm_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:26.452 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:26.722 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-18T00:55:26.738 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-18T00:55:26.749 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:26.765 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-18T00:55:26.796 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-18T00:55:26.797 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:27.107 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-18T00:55:27.108 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-18T00:55:27.123 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-18T00:55:27.135 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-18T00:55:27.157 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-18T00:55:27.166 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-18T00:55:27.461 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-routes. 2024-09-18T00:55:27.488 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-18T00:55:27.528 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-18T00:55:27.814 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-18T00:55:27.841 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:27.881 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-dashboard (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-babel. 2024-09-18T00:55:28.440 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-18T00:55:28.487 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-18T00:55:28.765 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-jinja2. 2024-09-18T00:55:28.781 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-18T00:55:28.823 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-18T00:55:29.185 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-18T00:55:29.212 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:29.251 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:29.378 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-18T00:55:29.405 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-18T00:55:29.444 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-18T00:55:29.580 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-18T00:55:29.607 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-18T00:55:29.646 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-18T00:55:29.890 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-routes. 2024-09-18T00:55:29.906 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-18T00:55:29.940 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-18T00:55:29.948 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-joblib. 2024-09-18T00:55:29.975 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-18T00:55:30.014 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-18T00:55:30.252 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-18T00:55:30.279 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:30.318 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-dashboard (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:30.376 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-18T00:55:30.403 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-18T00:55:30.442 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-18T00:55:30.737 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn. 2024-09-18T00:55:30.764 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-18T00:55:30.803 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-18T00:55:31.504 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-18T00:55:31.531 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:31.590 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-18T00:55:31.607 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-18T00:55:31.648 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-18T00:55:31.655 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:32.127 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-joblib. 2024-09-18T00:55:32.145 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-18T00:55:32.177 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-18T00:55:32.522 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-18T00:55:32.549 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-18T00:55:32.588 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-18T00:55:32.625 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cachetools. 2024-09-18T00:55:32.641 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-18T00:55:32.675 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-18T00:55:32.850 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-sklearn. 2024-09-18T00:55:32.877 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-18T00:55:32.916 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-18T00:55:32.970 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-18T00:55:32.997 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-18T00:55:33.036 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-18T00:55:33.365 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-18T00:55:33.392 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-18T00:55:33.431 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-18T00:55:33.641 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-18T00:55:33.669 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:33.717 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rsa. 2024-09-18T00:55:33.744 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-18T00:55:33.784 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-18T00:55:33.793 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:34.096 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-google-auth. 2024-09-18T00:55:34.123 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-18T00:55:34.162 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-18T00:55:34.465 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-18T00:55:34.481 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-18T00:55:34.515 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-18T00:55:34.755 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-cachetools. 2024-09-18T00:55:34.782 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-18T00:55:34.810 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-websocket. 2024-09-18T00:55:34.821 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-18T00:55:34.838 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-18T00:55:34.876 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-18T00:55:35.074 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-18T00:55:35.099 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-18T00:55:35.132 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-18T00:55:35.197 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-18T00:55:35.224 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-18T00:55:35.263 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-18T00:55:35.494 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-18T00:55:35.522 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-18T00:55:35.561 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-18T00:55:35.856 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-rsa. 2024-09-18T00:55:35.883 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-18T00:55:35.918 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-18T00:55:35.922 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-18T00:55:35.935 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:35.977 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:36.217 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-google-auth. 2024-09-18T00:55:36.245 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-18T00:55:36.284 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-18T00:55:36.288 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-18T00:55:36.316 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-18T00:55:36.354 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-18T00:55:36.562 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-18T00:55:36.590 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-18T00:55:36.628 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-18T00:55:36.682 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-18T00:55:36.710 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-18T00:55:36.749 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-18T00:55:36.940 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-websocket. 2024-09-18T00:55:36.968 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-18T00:55:36.993 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package jq. 2024-09-18T00:55:37.006 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-18T00:55:37.009 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-18T00:55:37.051 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-18T00:55:37.296 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package socat. 2024-09-18T00:55:37.312 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-18T00:55:37.326 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-18T00:55:37.345 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-18T00:55:37.354 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-18T00:55:37.392 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-18T00:55:37.641 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package xmlstarlet. 2024-09-18T00:55:37.656 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-18T00:55:37.699 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-18T00:55:38.082 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-18T00:55:38.095 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-test. 2024-09-18T00:55:38.099 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:38.123 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../097-ceph-test_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:38.131 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:38.161 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-test (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:38.435 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-18T00:55:38.462 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-18T00:55:38.502 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-18T00:55:38.821 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-18T00:55:38.849 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-18T00:55:38.887 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-18T00:55:39.173 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package jq. 2024-09-18T00:55:39.190 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-18T00:55:39.231 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-18T00:55:39.459 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package socat. 2024-09-18T00:55:39.476 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-18T00:55:39.509 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-18T00:55:39.813 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package xmlstarlet. 2024-09-18T00:55:39.841 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-18T00:55:39.879 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-18T00:55:40.216 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-test. 2024-09-18T00:55:40.233 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../097-ceph-test_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:40.266 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-test (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:40.474 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-volume. 2024-09-18T00:55:40.502 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:40.540 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-volume (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:40.844 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-18T00:55:40.872 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:40.919 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs-dev (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:41.264 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-18T00:55:41.291 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-18T00:55:41.330 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-18T00:55:41.660 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-18T00:55:41.688 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-18T00:55:41.726 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-18T00:55:42.089 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package nvme-cli. 2024-09-18T00:55:42.117 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-18T00:55:42.155 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-18T00:55:42.577 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package pkg-config. 2024-09-18T00:55:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-18T00:55:42.643 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-18T00:55:42.663 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package ceph-volume. 2024-09-18T00:55:42.691 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:55:42.729 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking ceph-volume (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:42.906 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-18T00:55:42.922 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-18T00:55:42.965 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-18T00:55:43.049 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-18T00:55:43.066 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:43.107 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking libcephfs-dev (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:43.368 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-18T00:55:43.396 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-18T00:55:43.434 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-18T00:55:43.794 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-18T00:55:43.822 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-18T00:55:43.868 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-18T00:55:43.895 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastescript. 2024-09-18T00:55:43.923 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-18T00:55:43.962 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-18T00:55:44.198 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-18T00:55:44.227 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-18T00:55:44.265 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-18T00:55:44.282 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pluggy. 2024-09-18T00:55:44.310 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-18T00:55:44.348 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-18T00:55:44.652 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package nvme-cli. 2024-09-18T00:55:44.660 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-psutil. 2024-09-18T00:55:44.680 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-18T00:55:44.688 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-18T00:55:44.718 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-18T00:55:44.718 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-18T00:55:45.055 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-py. 2024-09-18T00:55:45.083 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-18T00:55:45.120 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-18T00:55:45.132 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package pkg-config. 2024-09-18T00:55:45.161 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-18T00:55:45.199 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-18T00:55:45.424 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pygments. 2024-09-18T00:55:45.452 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-18T00:55:45.460 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-18T00:55:45.488 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-18T00:55:45.490 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-18T00:55:45.526 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-18T00:55:45.928 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-18T00:55:45.955 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-18T00:55:45.973 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-18T00:55:45.994 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-18T00:55:46.002 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-18T00:55:46.040 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-18T00:55:46.280 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-toml. 2024-09-18T00:55:46.308 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-18T00:55:46.326 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pastescript. 2024-09-18T00:55:46.347 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-18T00:55:46.354 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-18T00:55:46.392 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-18T00:55:46.650 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pytest. 2024-09-18T00:55:46.677 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-18T00:55:46.687 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pluggy. 2024-09-18T00:55:46.704 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-18T00:55:46.716 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-18T00:55:46.737 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-18T00:55:47.032 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-psutil. 2024-09-18T00:55:47.053 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplejson. 2024-09-18T00:55:47.060 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-18T00:55:47.080 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-18T00:55:47.098 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-18T00:55:47.121 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-18T00:55:47.389 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-18T00:55:47.416 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-18T00:55:47.435 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-py. 2024-09-18T00:55:47.455 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-18T00:55:47.463 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-18T00:55:47.501 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-18T00:55:47.779 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pygments. 2024-09-18T00:55:47.808 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-18T00:55:47.845 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-18T00:55:48.134 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package radosgw. 2024-09-18T00:55:48.162 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../116-radosgw_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:48.192 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking radosgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:48.300 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-18T00:55:48.329 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-18T00:55:48.366 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-18T00:55:48.694 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-toml. 2024-09-18T00:55:48.723 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-18T00:55:48.760 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-18T00:55:48.970 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package rbd-fuse. 2024-09-18T00:55:48.987 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:49.028 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking rbd-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:49.063 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-pytest. 2024-09-18T00:55:49.093 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-18T00:55:49.130 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-18T00:55:49.418 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-18T00:55:49.475 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package python3-simplejson. 2024-09-18T00:55:49.503 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-18T00:55:49.541 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-18T00:55:49.600 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-18T00:55:49.726 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-18T00:55:49.811 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-18T00:55:49.828 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-18T00:55:49.869 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-18T00:55:49.992 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-18T00:55:50.322 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-18T00:55:50.607 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package radosgw. 2024-09-18T00:55:50.635 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../116-radosgw_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:50.673 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking radosgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:50.746 INFO:teuthology.orchestra.run.smithi195.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-18T00:55:50.811 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:51.369 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user cephadm....done 2024-09-18T00:55:51.493 INFO:teuthology.orchestra.run.smithi042.stdout:Selecting previously unselected package rbd-fuse. 2024-09-18T00:55:51.521 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:55:51.525 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-18T00:55:51.550 INFO:teuthology.orchestra.run.smithi042.stdout:Unpacking rbd-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:51.793 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-18T00:55:51.839 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-18T00:55:52.053 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-18T00:55:52.089 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-18T00:55:52.162 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-18T00:55:52.207 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-18T00:55:52.414 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-18T00:55:52.468 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-18T00:55:52.673 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-18T00:55:52.796 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-18T00:55:52.799 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-18T00:55:53.101 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-18T00:55:53.237 INFO:teuthology.orchestra.run.smithi042.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-18T00:55:53.461 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-18T00:55:53.496 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:53.695 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-18T00:55:53.963 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-18T00:55:54.034 INFO:teuthology.orchestra.run.smithi042.stdout:Adding system user cephadm....done 2024-09-18T00:55:54.174 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-18T00:55:54.273 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-18T00:55:54.459 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-18T00:55:54.558 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-18T00:55:54.701 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-18T00:55:54.817 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-18T00:55:54.827 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-18T00:55:55.087 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-18T00:55:55.168 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-argparse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:55.346 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-18T00:55:55.419 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-18T00:55:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-18T00:55:55.537 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-18T00:55:55.638 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-18T00:55:55.747 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-18T00:55:55.791 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-18T00:55:55.998 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-18T00:55:56.151 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-18T00:55:56.411 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-18T00:55:56.476 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-18T00:55:56.504 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-18T00:55:56.568 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-18T00:55:56.669 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-18T00:55:56.687 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-18T00:55:56.939 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-18T00:55:57.020 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-18T00:55:57.121 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-18T00:55:57.198 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-18T00:55:57.389 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-18T00:55:57.465 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-18T00:55:57.657 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-18T00:55:57.766 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-ceph-argparse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:55:57.975 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up zip (3.0-12build2) ... 2024-09-18T00:55:58.009 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-18T00:55:58.110 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-18T00:55:58.118 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-18T00:55:58.244 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-18T00:55:58.370 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-18T00:55:58.638 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-18T00:55:58.811 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-18T00:55:59.045 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-18T00:55:59.123 INFO:teuthology.orchestra.run.smithi042.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-18T00:55:59.171 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-18T00:55:59.297 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-18T00:55:59.378 INFO:teuthology.orchestra.run.smithi042.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-18T00:55:59.441 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-18T00:55:59.542 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-18T00:55:59.598 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-18T00:55:59.910 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-18T00:56:00.000 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-18T00:56:00.036 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-18T00:56:00.259 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-18T00:56:00.304 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-18T00:56:00.572 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-18T00:56:00.635 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-18T00:56:00.890 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up zip (3.0-12build2) ... 2024-09-18T00:56:00.928 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-18T00:56:01.016 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-18T00:56:01.263 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-18T00:56:01.522 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-18T00:56:01.649 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:01.726 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-18T00:56:01.941 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-18T00:56:01.977 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-18T00:56:02.094 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-18T00:56:02.195 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-18T00:56:02.268 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-18T00:56:02.488 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-18T00:56:02.923 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-18T00:56:03.182 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-18T00:56:03.406 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-18T00:56:03.575 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-18T00:56:03.658 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:56:03.793 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-18T00:56:03.868 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-18T00:56:04.060 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-18T00:56:04.186 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-18T00:56:04.220 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-18T00:56:04.287 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-18T00:56:04.471 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-18T00:56:04.572 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-18T00:56:04.588 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:04.815 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-18T00:56:04.898 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-18T00:56:05.066 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:56:05.191 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-18T00:56:05.249 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-18T00:56:05.485 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-18T00:56:05.744 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-18T00:56:05.995 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-18T00:56:06.238 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-18T00:56:06.454 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-18T00:56:06.489 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-18T00:56:06.723 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:56:06.732 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-18T00:56:06.841 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-18T00:56:06.857 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-18T00:56:07.117 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-18T00:56:07.158 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-18T00:56:07.243 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-18T00:56:07.259 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-18T00:56:07.368 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-18T00:56:07.544 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-18T00:56:07.687 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-18T00:56:07.870 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-18T00:56:07.929 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-18T00:56:08.130 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-18T00:56:08.181 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:56:08.256 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-18T00:56:08.306 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-18T00:56:08.382 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-18T00:56:08.472 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-18T00:56:08.625 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-18T00:56:08.717 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-18T00:56:08.843 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-18T00:56:08.875 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-18T00:56:09.168 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-18T00:56:09.228 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-18T00:56:09.444 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-18T00:56:09.487 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-18T00:56:09.613 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-18T00:56:09.696 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-18T00:56:09.873 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:56:09.988 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-18T00:56:09.999 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-18T00:56:10.114 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-18T00:56:10.325 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-18T00:56:10.399 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-18T00:56:10.516 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-18T00:56:10.634 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-18T00:56:10.817 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-18T00:56:10.894 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-18T00:56:11.135 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-18T00:56:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-18T00:56:11.461 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-18T00:56:11.504 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-18T00:56:11.613 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-18T00:56:11.703 INFO:teuthology.orchestra.run.smithi042.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-18T00:56:11.737 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-18T00:56:11.982 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-18T00:56:12.014 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-18T00:56:12.108 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-18T00:56:12.342 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-18T00:56:12.534 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-18T00:56:12.710 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-18T00:56:12.793 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-18T00:56:12.911 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-18T00:56:12.987 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-18T00:56:13.162 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:56:13.288 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-18T00:56:13.339 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-18T00:56:13.465 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librados2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:13.583 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libsqlite3-mod-ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:13.623 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-18T00:56:13.709 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-18T00:56:13.949 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-18T00:56:14.045 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:14.171 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libradosstriper1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:14.233 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-18T00:56:14.298 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-18T00:56:14.359 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-18T00:56:14.566 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librbd1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:14.644 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-18T00:56:14.692 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-modules-core (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:14.843 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:14.919 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-18T00:56:15.045 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T00:56:15.045 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T00:56:15.273 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-18T00:56:15.536 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs-dev (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:15.658 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-18T00:56:15.662 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rados (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:15.788 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librgw2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:15.914 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-18T00:56:15.926 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-18T00:56:16.320 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-18T00:56:16.446 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librados2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:16.573 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libsqlite3-mod-ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:16.715 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-18T00:56:17.026 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libcephfs2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:17.117 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rbd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:17.136 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libradosstriper1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:17.226 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up rbd-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:17.262 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-18T00:56:17.344 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:17.470 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cephfs (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:17.538 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librbd1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:17.605 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:17.664 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-modules-core (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:17.790 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:18.009 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T00:56:18.009 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T00:56:18.181 INFO:teuthology.orchestra.run.smithi195.stdout:Adding group ceph....done 2024-09-18T00:56:18.490 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user ceph....done 2024-09-18T00:56:18.517 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up libcephfs-dev (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:18.618 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rados (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:18.651 INFO:teuthology.orchestra.run.smithi195.stdout:Setting system user ceph properties....done 2024-09-18T00:56:18.668 INFO:teuthology.orchestra.run.smithi195.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-18T00:56:18.718 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up librgw2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:18.811 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-18T00:56:18.836 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-18T00:56:19.169 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-18T00:56:19.631 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-test (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:19.757 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up radosgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:20.090 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rbd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:20.216 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up rbd-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:20.232 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T00:56:20.232 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T00:56:20.342 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-rgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:20.468 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up python3-cephfs (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:20.594 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:20.692 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-base (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:21.040 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-18T00:56:21.112 INFO:teuthology.orchestra.run.smithi042.stdout:Adding group ceph....done 2024-09-18T00:56:21.485 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mds (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:21.489 INFO:teuthology.orchestra.run.smithi042.stdout:Adding system user ceph....done 2024-09-18T00:56:21.657 INFO:teuthology.orchestra.run.smithi042.stdout:Setting system user ceph properties....done 2024-09-18T00:56:21.676 INFO:teuthology.orchestra.run.smithi042.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-18T00:56:21.705 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T00:56:21.705 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T00:56:21.819 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-18T00:56:22.154 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:22.161 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-18T00:56:22.373 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T00:56:22.373 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T00:56:22.703 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-test (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:22.822 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-osd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:22.829 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up radosgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:23.095 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T00:56:23.096 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T00:56:23.318 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T00:56:23.319 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T00:56:23.582 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-k8sevents (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:23.708 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:23.790 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-base (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:23.859 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mon (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:24.073 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T00:56:24.073 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T00:56:24.152 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-18T00:56:24.568 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:24.659 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mds (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:24.719 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:24.862 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-dashboard (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T00:56:24.872 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T00:56:24.997 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-volume (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:25.400 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-18T00:56:25.500 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-18T00:56:25.576 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T00:56:25.576 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T00:56:25.653 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-18T00:56:26.045 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-osd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:26.324 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T00:56:26.324 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T00:56:26.779 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-k8sevents (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:26.797 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-18T00:56:26.914 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:27.040 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mon (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:27.240 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T00:56:27.240 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T00:56:27.446 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-18T00:56:27.452 DEBUG:teuthology.parallel:result is None 2024-09-18T00:56:27.716 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:27.850 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:27.976 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-mgr-dashboard (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:28.127 INFO:teuthology.orchestra.run.smithi042.stdout:Setting up ceph-volume (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:56:28.489 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-18T00:56:28.606 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-18T00:56:28.742 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-18T00:56:30.244 INFO:teuthology.orchestra.run.smithi042.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-18T00:56:30.924 INFO:teuthology.orchestra.run.smithi042.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-18T00:56:30.929 DEBUG:teuthology.parallel:result is None 2024-09-18T00:56:30.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:56:31.072 DEBUG:teuthology.orchestra.run.smithi042:> dpkg-query -W -f '${Version}' ceph 2024-09-18T00:56:31.091 INFO:teuthology.orchestra.run.smithi042.stdout:19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:56:31.092 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:56:31.092 INFO:teuthology.task.install:The correct ceph version 19.1.1-330-gc2ddf972-1jammy is installed. 2024-09-18T00:56:31.094 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:56:31.231 DEBUG:teuthology.orchestra.run.smithi195:> dpkg-query -W -f '${Version}' ceph 2024-09-18T00:56:31.250 INFO:teuthology.orchestra.run.smithi195.stdout:19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:56:31.250 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:56:31.250 INFO:teuthology.task.install:The correct ceph version 19.1.1-330-gc2ddf972-1jammy is installed. 2024-09-18T00:56:31.252 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-18T00:56:31.252 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:56:31.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T00:56:31.267 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:56:31.267 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T00:56:31.308 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-18T00:56:31.308 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:56:31.309 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T00:56:31.322 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T00:56:31.377 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:56:31.377 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T00:56:31.390 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T00:56:31.450 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-18T00:56:31.450 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:56:31.451 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T00:56:31.464 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T00:56:31.518 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:56:31.518 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T00:56:31.530 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T00:56:31.586 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-18T00:56:31.586 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:56:31.587 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T00:56:31.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T00:56:31.657 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:56:31.657 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T00:56:31.670 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T00:56:31.726 INFO:teuthology.run_tasks:Running task ceph... 2024-09-18T00:56:31.813 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-18T00:56:31.813 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /var/log/ceph 2024-09-18T00:56:31.815 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /var/log/ceph 2024-09-18T00:56:31.828 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-18T00:56:31.828 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T00:56:31.871 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T00:56:31.883 INFO:tasks.ceph:Creating extra log directories... 2024-09-18T00:56:31.883 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T00:56:31.927 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T00:56:31.952 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-18T00:56:31.952 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-18T00:56:31.953 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909680', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1983, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'sleep_before_teardown': 0, 'subset': '79/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7', 'targets': {'smithi042.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDdmUOl4ryUlfP/sCj0rAvvbzPuu4kPaUquCAH01QG7QHP+G5eLIdDDp+yQj9z4KKmsqR1Wd3X5HqxafhC6b5mo=', 'smithi195.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAXWVER4b9q8kCr3qUCuRi6Qv0nTyEZlr8rhOHtyTWFOLkj+watrKaSAZQYUG526cGmaEgMv28zbrD3BXosoroI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-18T00:56:31.953 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T00:56:31.983 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T00:56:31.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T00:56:32.031 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T00:56:32.050 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:56:32.051 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T00:56:32.090 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T00:56:32.090 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-09-18T00:56:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T00:56:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:56:32.139 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 875 Links: 1 2024-09-18T00:56:32.139 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:56:32.139 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-18 00:51:04.547702300 +0000 2024-09-18T00:56:32.139 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-18 00:51:04.267704800 +0000 2024-09-18T00:56:32.139 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-18 00:51:04.267704800 +0000 2024-09-18T00:56:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-09-18T00:56:32.139 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T00:56:32.193 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-18T00:56:32.193 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-18T00:56:32.193 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.00032454 s, 1.6 MB/s 2024-09-18T00:56:32.194 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T00:56:32.244 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-09-18T00:56:32.290 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T00:56:32.290 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:56:32.290 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 897 Links: 1 2024-09-18T00:56:32.290 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:56:32.291 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-18 00:51:05.067697654 +0000 2024-09-18T00:56:32.291 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-18 00:51:04.795700085 +0000 2024-09-18T00:56:32.291 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-18 00:51:04.795700085 +0000 2024-09-18T00:56:32.291 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-09-18T00:56:32.291 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T00:56:32.344 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-18T00:56:32.344 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-18T00:56:32.345 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000259946 s, 2.0 MB/s 2024-09-18T00:56:32.346 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T00:56:32.396 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-09-18T00:56:32.446 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T00:56:32.446 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:56:32.447 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-09-18T00:56:32.447 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:56:32.447 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-18 00:51:05.303695546 +0000 2024-09-18T00:56:32.447 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-18 00:51:05.299695583 +0000 2024-09-18T00:56:32.447 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-18 00:51:05.299695583 +0000 2024-09-18T00:56:32.447 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-09-18T00:56:32.447 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T00:56:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-18T00:56:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-18T00:56:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000314788 s, 1.6 MB/s 2024-09-18T00:56:32.506 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T00:56:32.555 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-09-18T00:56:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T00:56:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:56:32.602 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-09-18T00:56:32.602 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:56:32.602 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-18 00:51:06.051688866 +0000 2024-09-18T00:56:32.602 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-18 00:51:05.795691152 +0000 2024-09-18T00:56:32.603 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-18 00:51:05.795691152 +0000 2024-09-18T00:56:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-09-18T00:56:32.603 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T00:56:32.656 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-18T00:56:32.657 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-18T00:56:32.657 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000303866 s, 1.7 MB/s 2024-09-18T00:56:32.658 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T00:56:32.707 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T00:56:32.708 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:56:32.708 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T00:56:32.713 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T00:56:32.714 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-09-18T00:56:32.760 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T00:56:32.760 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:56:32.760 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 880 Links: 1 2024-09-18T00:56:32.760 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:56:32.760 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-18 00:51:03.249362911 +0000 2024-09-18T00:56:32.760 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-18 00:51:02.985365257 +0000 2024-09-18T00:56:32.761 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-18 00:51:02.985365257 +0000 2024-09-18T00:56:32.761 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-09-18T00:56:32.761 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T00:56:32.813 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-18T00:56:32.813 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-18T00:56:32.813 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000320128 s, 1.6 MB/s 2024-09-18T00:56:32.815 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T00:56:32.866 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-09-18T00:56:32.915 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T00:56:32.915 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:56:32.915 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 898 Links: 1 2024-09-18T00:56:32.915 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:56:32.915 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-18 00:51:03.733358611 +0000 2024-09-18T00:56:32.915 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-18 00:51:03.473360921 +0000 2024-09-18T00:56:32.915 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-18 00:51:03.473360921 +0000 2024-09-18T00:56:32.916 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-09-18T00:56:32.916 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T00:56:32.968 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-18T00:56:32.968 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-18T00:56:32.968 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000199117 s, 2.6 MB/s 2024-09-18T00:56:32.970 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T00:56:33.020 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-09-18T00:56:33.067 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T00:56:33.067 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:56:33.068 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 913 Links: 1 2024-09-18T00:56:33.068 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:56:33.068 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-18 00:51:03.981356408 +0000 2024-09-18T00:56:33.068 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-18 00:51:03.977356444 +0000 2024-09-18T00:56:33.068 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-18 00:51:03.977356444 +0000 2024-09-18T00:56:33.068 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-09-18T00:56:33.068 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T00:56:33.120 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-18T00:56:33.120 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-18T00:56:33.121 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000201536 s, 2.5 MB/s 2024-09-18T00:56:33.122 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T00:56:33.168 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-09-18T00:56:33.215 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T00:56:33.215 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:56:33.215 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 930 Links: 1 2024-09-18T00:56:33.215 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:56:33.215 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-18 00:51:04.681350190 +0000 2024-09-18T00:56:33.215 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-18 00:51:04.433352393 +0000 2024-09-18T00:56:33.215 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-18 00:51:04.433352393 +0000 2024-09-18T00:56:33.215 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-09-18T00:56:33.216 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T00:56:33.269 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-18T00:56:33.269 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-18T00:56:33.269 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000265508 s, 1.9 MB/s 2024-09-18T00:56:33.270 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T00:56:33.321 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T00:56:33.321 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T00:56:33.321 INFO:tasks.ceph:Generating config... 2024-09-18T00:56:33.322 INFO:tasks.ceph:[client] rbd cache = True 2024-09-18T00:56:33.322 INFO:tasks.ceph:[client] rbd clone copy on read = True 2024-09-18T00:56:33.322 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-18T00:56:33.322 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-18T00:56:33.323 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-18T00:56:33.323 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-18T00:56:33.323 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-18T00:56:33.323 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-18T00:56:33.323 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-18T00:56:33.323 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-18T00:56:33.323 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-18T00:56:33.323 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-18T00:56:33.323 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-18T00:56:33.323 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-18T00:56:33.323 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-18T00:56:33.323 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-18T00:56:33.323 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-18T00:56:33.324 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-18T00:56:33.324 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-18T00:56:33.324 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-18T00:56:33.324 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-18T00:56:33.324 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-18T00:56:33.324 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-18T00:56:33.324 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-18T00:56:33.324 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-18T00:56:33.324 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-18T00:56:33.324 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-18T00:56:33.324 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-18T00:56:33.324 INFO:tasks.ceph:Setting up mon.a... 2024-09-18T00:56:33.325 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-18T00:56:33.360 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.keyring 2024-09-18T00:56:33.377 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-18T00:56:33.460 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T00:56:33.513 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.42'), ('mon.c', '[v2:172.21.15.42:3301,v1:172.21.15.42:6790]'), ('mon.b', '172.21.15.195')] 2024-09-18T00:56:33.514 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.195', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd clone copy on read': True}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-18T00:56:33.514 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:56:33.515 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T00:56:33.562 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.42 --addv c '[v2:172.21.15.42:3301,v1:172.21.15.42:6790]' --add b 172.21.15.195 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:56:33.649 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:56:33.650 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: generated fsid 82a96d76-ba83-49e2-a985-84604bd599ba 2024-09-18T00:56:33.650 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = quincy 2024-09-18T00:56:33.650 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 0 2024-09-18T00:56:33.650 INFO:teuthology.orchestra.run.smithi042.stdout:fsid 82a96d76-ba83-49e2-a985-84604bd599ba 2024-09-18T00:56:33.650 INFO:teuthology.orchestra.run.smithi042.stdout:last_changed 2024-09-18T00:56:33.649830+0000 2024-09-18T00:56:33.650 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-18T00:56:33.649830+0000 2024-09-18T00:56:33.650 INFO:teuthology.orchestra.run.smithi042.stdout:min_mon_release 17 (quincy) 2024-09-18T00:56:33.650 INFO:teuthology.orchestra.run.smithi042.stdout:election_strategy: 1 2024-09-18T00:56:33.650 INFO:teuthology.orchestra.run.smithi042.stdout:0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-09-18T00:56:33.650 INFO:teuthology.orchestra.run.smithi042.stdout:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-09-18T00:56:33.651 INFO:teuthology.orchestra.run.smithi042.stdout:2: [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] mon.c 2024-09-18T00:56:33.651 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-18T00:56:33.652 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T00:56:33.656 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 82a96d76-ba83-49e2-a985-84604bd599ba... 2024-09-18T00:56:33.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T00:56:33.699 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T00:56:33.728 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-09-18T00:56:33.728 INFO:teuthology.orchestra.run.smithi042.stdout: chdir = "" 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: auth supported = cephx 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: filestore xattr use omap = true 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: mon clock drift allowed = 1.000 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: osd crush chooseleaf type = 0 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: auth debug = true 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on old message = true 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on bug = true 2024-09-18T00:56:33.729 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.730 INFO:teuthology.orchestra.run.smithi042.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T00:56:33.730 INFO:teuthology.orchestra.run.smithi042.stdout: mon pg warn max object skew = 0 2024-09-18T00:56:33.730 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.730 INFO:teuthology.orchestra.run.smithi042.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T00:56:33.730 INFO:teuthology.orchestra.run.smithi042.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T00:56:33.730 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.730 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default size = 2 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow primary affinity = true 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow pg remap = true 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on legacy crush tunables = false 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on crush straw calc version zero = false 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on no sortbitwise = false 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on osd down out interval zero = false 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on too few osds = false 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: mon_allow_pool_size_one = true 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.731 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T00:56:33.732 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.732 INFO:teuthology.orchestra.run.smithi042.stdout: osd default data pool replay window = 5 2024-09-18T00:56:33.732 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.732 INFO:teuthology.orchestra.run.smithi042.stdout: mon allow pool delete = true 2024-09-18T00:56:33.732 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.732 INFO:teuthology.orchestra.run.smithi042.stdout: mon cluster log file level = debug 2024-09-18T00:56:33.732 INFO:teuthology.orchestra.run.smithi042.stdout: debug asserts on shutdown = true 2024-09-18T00:56:33.732 INFO:teuthology.orchestra.run.smithi042.stdout: mon health detail to clog = false 2024-09-18T00:56:33.732 INFO:teuthology.orchestra.run.smithi042.stdout: mon host = "172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.195" 2024-09-18T00:56:33.732 INFO:teuthology.orchestra.run.smithi042.stdout: mon client directed command retry = 5 2024-09-18T00:56:33.732 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on pool no app = False 2024-09-18T00:56:33.733 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject socket failures = 5000 2024-09-18T00:56:33.733 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = 82a96d76-ba83-49e2-a985-84604bd599ba 2024-09-18T00:56:33.733 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.733 INFO:teuthology.orchestra.run.smithi042.stdout:[osd] 2024-09-18T00:56:33.733 INFO:teuthology.orchestra.run.smithi042.stdout: osd journal size = 100 2024-09-18T00:56:33.733 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.733 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub load threshold = 5.0 2024-09-18T00:56:33.733 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub max interval = 600 2024-09-18T00:56:33.733 INFO:teuthology.orchestra.run.smithi042.stdout: osd mclock profile = high_recovery_ops 2024-09-18T00:56:33.733 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: osd recover clone overlap = true 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: osd recovery max chunk = 1048576 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug shutdown = true 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug op order = true 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify stray on activate = true 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug trim objects = true 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: osd open classes on start = true 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug pg log writeout = true 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: osd deep scrub update digest min age = 30 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.734 INFO:teuthology.orchestra.run.smithi042.stdout: osd map max advance = 10 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: journal zero on create = true 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: filestore ondisk finisher threads = 3 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: filestore apply finisher threads = 3 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: bdev debug aio = true 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug misdirected ops = true 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: bdev async discard = True 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: bdev enable discard = True 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore block size = 96636764160 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore fsck on mount = True 2024-09-18T00:56:33.735 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluefs = 1/20 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluestore = 1/20 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: debug osd = 20 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: debug rocksdb = 4/10 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd full ratio = 0.9 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd nearfull ratio = 0.8 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: osd blocked scrub grace period = 3600 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: osd failsafe full ratio = 0.95 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: osd max markdown count = 1000 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: osd memory target = 1610612736 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: osd objectstore = bluestore 2024-09-18T00:56:33.736 INFO:teuthology.orchestra.run.smithi042.stdout: osd shutdown pgref assert = True 2024-09-18T00:56:33.737 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.737 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr] 2024-09-18T00:56:33.737 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-18T00:56:33.737 INFO:teuthology.orchestra.run.smithi042.stdout: debug mgr = 20 2024-09-18T00:56:33.737 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-09-18T00:56:33.737 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-09-18T00:56:33.737 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-09-18T00:56:33.737 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-09-18T00:56:33.737 INFO:teuthology.orchestra.run.smithi042.stdout: mgr/telemetry/nag = false 2024-09-18T00:56:33.737 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.737 INFO:teuthology.orchestra.run.smithi042.stdout:[mon] 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: debug paxos = 20 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: mon data avail warn = 5 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: mon mgr mkfs grace = 240 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd reporter subtree level = osd 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd prime pg temp = true 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.738 INFO:teuthology.orchestra.run.smithi042.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: # 1m isn't quite enough 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: mon_down_mkfs_grace = 2m 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_filestore_osds = false 2024-09-18T00:56:33.739 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:56:33.740 INFO:teuthology.orchestra.run.smithi042.stdout:[client] 2024-09-18T00:56:33.740 INFO:teuthology.orchestra.run.smithi042.stdout: rgw cache enabled = true 2024-09-18T00:56:33.740 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable ops log = true 2024-09-18T00:56:33.740 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable usage log = true 2024-09-18T00:56:33.740 INFO:teuthology.orchestra.run.smithi042.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T00:56:33.740 INFO:teuthology.orchestra.run.smithi042.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T00:56:33.740 INFO:teuthology.orchestra.run.smithi042.stdout: rbd cache = True 2024-09-18T00:56:33.740 INFO:teuthology.orchestra.run.smithi042.stdout: rbd clone copy on read = True 2024-09-18T00:56:33.740 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.a] 2024-09-18T00:56:33.740 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.c] 2024-09-18T00:56:33.740 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.b] 2024-09-18T00:56:33.742 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-09-18T00:56:33.742 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-09-18T00:56:33.742 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T00:56:33.742 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-09-18T00:56:33.742 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.743 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-09-18T00:56:33.743 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.743 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-09-18T00:56:33.743 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.743 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-09-18T00:56:33.743 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-09-18T00:56:33.743 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.743 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-09-18T00:56:33.743 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-09-18T00:56:33.743 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.743 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-09-18T00:56:33.744 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.745 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.195" 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: mon client directed command retry = 5 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on pool no app = False 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject socket failures = 5000 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 82a96d76-ba83-49e2-a985-84604bd599ba 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-09-18T00:56:33.746 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug trim objects = true 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-09-18T00:56:33.747 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-09-18T00:56:33.748 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 1/20 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 1/20 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 4/10 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-09-18T00:56:33.749 INFO:teuthology.orchestra.run.smithi195.stdout: osd blocked scrub grace period = 3600 2024-09-18T00:56:33.750 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-09-18T00:56:33.750 INFO:teuthology.orchestra.run.smithi195.stdout: osd max markdown count = 1000 2024-09-18T00:56:33.750 INFO:teuthology.orchestra.run.smithi195.stdout: osd memory target = 1610612736 2024-09-18T00:56:33.750 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-09-18T00:56:33.750 INFO:teuthology.orchestra.run.smithi195.stdout: osd shutdown pgref assert = True 2024-09-18T00:56:33.750 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.750 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-09-18T00:56:33.750 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-18T00:56:33.750 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-09-18T00:56:33.750 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-09-18T00:56:33.750 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-09-18T00:56:33.751 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-09-18T00:56:33.752 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout: rbd cache = True 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout: rbd clone copy on read = True 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-09-18T00:56:33.753 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-09-18T00:56:33.754 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-18T00:56:33.754 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-18T00:56:33.835 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-18T00:56:33.835 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:56:33.835 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-18T00:56:33.842 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:56:33.842 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-18T00:56:33.886 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi042.front.sepia.ceph.com 2024-09-18T00:56:33.887 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:56:33.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T00:56:33.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T00:56:33.951 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:56:33.951 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:56:33.998 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi195.front.sepia.ceph.com 2024-09-18T00:56:33.998 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:56:33.999 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T00:56:33.999 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T00:56:34.022 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:56:34.022 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:56:34.071 INFO:tasks.ceph:Setting up mon nodes... 2024-09-18T00:56:34.071 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-18T00:56:34.071 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T00:56:34.116 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T00:56:34.119 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T00:56:34.163 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T00:56:34.173 INFO:tasks.ceph:Setting up mds nodes... 2024-09-18T00:56:34.173 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-18T00:56:34.173 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-18T00:56:34.209 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-18T00:56:34.220 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-18T00:56:34.257 INFO:teuthology.orchestra.run.smithi195.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-18T00:56:34.266 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-18T00:56:34.266 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T00:56:34.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-18T00:56:34.279 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T00:56:34.279 INFO:tasks.ceph:role: osd.0 2024-09-18T00:56:34.279 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-18T00:56:34.279 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T00:56:34.335 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:56:34.335 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:56:34.335 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:56:34.335 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:56:34.335 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:56:34.335 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:56:34.335 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:56:34.335 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:56:34.335 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:56:34.335 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:56:34.362 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-18T00:56:34.364 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-18T00:56:34.364 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-18T00:56:34.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-18T00:56:34.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:56:34.475 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-18T00:56:34.528 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T00:56:34.528 INFO:tasks.ceph:role: osd.1 2024-09-18T00:56:34.528 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-18T00:56:34.528 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T00:56:34.582 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:56:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:56:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:56:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:56:34.583 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:56:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:56:34.583 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:56:34.583 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:56:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:56:34.583 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:56:34.609 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-18T00:56:34.610 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-18T00:56:34.610 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-18T00:56:34.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-18T00:56:34.722 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:56:34.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:56:34.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-18T00:56:34.776 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T00:56:34.776 INFO:tasks.ceph:role: osd.2 2024-09-18T00:56:34.776 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-18T00:56:34.776 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T00:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-18T00:56:34.858 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-18T00:56:34.858 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-18T00:56:34.916 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-18T00:56:34.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:56:34.968 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:56:34.968 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-18T00:56:35.026 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T00:56:35.026 INFO:tasks.ceph:role: osd.3 2024-09-18T00:56:35.026 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-18T00:56:35.027 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T00:56:35.081 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:56:35.081 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:56:35.081 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:56:35.081 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:56:35.081 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:56:35.081 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:56:35.082 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:56:35.082 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:56:35.082 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:56:35.082 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:56:35.108 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-18T00:56:35.110 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-18T00:56:35.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-18T00:56:35.170 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-18T00:56:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:56:35.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:56:35.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:56:35.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:35.355+0000 7f2f0553a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-18T00:56:35.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:35.355+0000 7f2f0553a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-18T00:56:35.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:35.355+0000 7f2f0553a7c0 -1 bdev(0x564ae9f1d800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-18T00:56:35.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:35.355+0000 7f2f0553a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-18T00:56:38.966 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-18T00:56:38.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:56:39.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:39.062+0000 7f09ebd1c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-18T00:56:39.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:39.062+0000 7f09ebd1c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-18T00:56:39.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:39.062+0000 7f09ebd1c7c0 -1 bdev(0x556effcc7800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-18T00:56:39.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:39.062+0000 7f09ebd1c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-18T00:56:42.466 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-18T00:56:42.523 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:56:42.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:42.602+0000 7f00c87037c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-18T00:56:42.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:42.602+0000 7f00c87037c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-18T00:56:42.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:42.602+0000 7f00c87037c0 -1 bdev(0x55d01201b800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-18T00:56:42.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:42.602+0000 7f00c87037c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-18T00:56:46.449 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-18T00:56:46.507 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:56:46.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:46.586+0000 7f07166767c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-18T00:56:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:46.586+0000 7f07166767c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-18T00:56:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:46.586+0000 7f07166767c0 -1 bdev(0x5633b96e7800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-18T00:56:46.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:56:46.586+0000 7f07166767c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-18T00:56:49.945 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-18T00:56:50.003 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-18T00:56:50.017 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T00:56:50.017 INFO:tasks.ceph:role: osd.4 2024-09-18T00:56:50.017 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-18T00:56:50.018 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T00:56:50.070 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:56:50.070 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:56:50.070 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:56:50.070 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:56:50.070 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:56:50.070 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:56:50.071 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:56:50.071 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:56:50.071 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:56:50.071 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:56:50.094 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-18T00:56:50.096 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-18T00:56:50.096 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-18T00:56:50.154 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-18T00:56:50.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:56:50.208 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:56:50.209 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-18T00:56:50.262 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T00:56:50.262 INFO:tasks.ceph:role: osd.5 2024-09-18T00:56:50.262 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-18T00:56:50.262 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T00:56:50.318 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:56:50.318 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:56:50.319 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:56:50.319 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:56:50.319 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:56:50.319 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:56:50.319 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:56:50.319 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:56:50.319 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:56:50.319 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:56:50.342 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-18T00:56:50.344 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-18T00:56:50.344 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-18T00:56:50.402 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-18T00:56:50.459 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:56:50.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:56:50.460 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-18T00:56:50.514 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T00:56:50.550 INFO:tasks.ceph:role: osd.6 2024-09-18T00:56:50.550 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-18T00:56:50.550 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T00:56:50.572 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:56:50.572 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:56:50.572 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:56:50.572 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:56:50.572 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:56:50.572 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:56:50.573 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:56:50.573 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:56:50.573 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:56:50.573 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:56:50.596 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-18T00:56:50.598 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-18T00:56:50.598 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-18T00:56:50.659 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-18T00:56:50.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:56:50.711 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:56:50.712 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-18T00:56:50.764 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T00:56:50.764 INFO:tasks.ceph:role: osd.7 2024-09-18T00:56:50.764 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-18T00:56:50.765 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T00:56:50.818 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:56:50.819 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:56:50.819 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:56:50.819 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:56:50.819 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:56:50.819 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:56:50.819 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:56:50.819 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:56:50.820 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:56:50.820 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:56:50.844 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-18T00:56:50.846 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-18T00:56:50.846 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-18T00:56:50.905 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-18T00:56:50.956 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:56:50.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:56:50.957 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:56:51.059 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:51.054+0000 7f76abde07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-18T00:56:51.060 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:51.054+0000 7f76abde07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-18T00:56:51.060 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:51.054+0000 7f76abde07c0 -1 bdev(0x561de63f7800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-18T00:56:51.060 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:51.054+0000 7f76abde07c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-18T00:56:54.468 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-18T00:56:54.481 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:56:54.560 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:54.554+0000 7f594265e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-18T00:56:54.561 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:54.554+0000 7f594265e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-18T00:56:54.561 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:54.554+0000 7f594265e7c0 -1 bdev(0x5617f6b05800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-18T00:56:54.561 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:54.554+0000 7f594265e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-18T00:56:57.934 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-18T00:56:57.996 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:56:58.078 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:58.070+0000 7f467b32d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-18T00:56:58.078 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:58.070+0000 7f467b32d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-18T00:56:58.078 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:58.070+0000 7f467b32d7c0 -1 bdev(0x562df5c9d800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-18T00:56:58.078 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:56:58.070+0000 7f467b32d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-18T00:57:01.438 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-18T00:57:01.495 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:57:01.576 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:57:01.570+0000 7f58eb1677c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-18T00:57:01.577 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:57:01.570+0000 7f58eb1677c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-18T00:57:01.577 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:57:01.570+0000 7f58eb1677c0 -1 bdev(0x5570c1323800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-18T00:57:01.577 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:57:01.570+0000 7f58eb1677c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-18T00:57:04.761 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-18T00:57:04.814 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-18T00:57:04.815 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:57:04.815 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-18T00:57:04.829 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:57:04.829 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-18T00:57:04.886 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:57:04.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-18T00:57:04.942 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:57:04.942 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-18T00:57:04.998 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:57:04.998 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-18T00:57:05.054 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:57:05.055 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-18T00:57:05.068 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:57:05.068 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-18T00:57:05.122 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:57:05.122 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-18T00:57:05.177 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:57:05.177 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-18T00:57:05.230 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:57:05.230 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-18T00:57:05.286 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:57:05.286 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-18T00:57:05.292 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:57:05.292 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-18T00:57:05.334 INFO:tasks.ceph:Adding keys to all mons... 2024-09-18T00:57:05.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T00:57:05.337 DEBUG:teuthology.orchestra.run.smithi195:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T00:57:05.381 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.y] 2024-09-18T00:57:05.381 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBCJepm08/pBhAAU+m4Qhr0k2Q3jjSdkPKP/Q== 2024-09-18T00:57:05.381 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.0] 2024-09-18T00:57:05.382 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBDJepmgARdFRAANYKMN+ZfR4AizHV0iyQ7zg== 2024-09-18T00:57:05.382 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.1] 2024-09-18T00:57:05.382 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBHJepmw1QIBBAAqq/q1IWTmb1Om8Vp88yCPQ== 2024-09-18T00:57:05.382 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.2] 2024-09-18T00:57:05.382 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBKJepmTWsvJBAAbPy2iYBpQwz9oaOUFymasQ== 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.3] 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBOJepmj9g/IxAAGr7gL5N5eX7/igYCgw+wYA== 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.x] 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBCJepmi/ylCRAA/EjXoM9tQOTDs3kB5874IQ== 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.4] 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBTJepmpSZ+AxAAMjfdfk7oHQfAQzp3HaXwdw== 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.5] 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBWJepmrLheIRAAw/ufOe0USo+dPFpOoWQ8HQ== 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.6] 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBaJepmVGSUBBAAY4ebUovOG6jHsfpVY7TSvg== 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.7] 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBdJepmCStTIhAAAZuWJrwvmzc2W0CliTTScw== 2024-09-18T00:57:05.383 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-09-18T00:57:05.384 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBCJepmF5t7DBAAOOrUAlQDk0egIiZJCQJmfA== 2024-09-18T00:57:05.384 INFO:teuthology.orchestra.run.smithi042.stdout:[client.1] 2024-09-18T00:57:05.384 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBCJepmvfM8DxAAbqiPFrGD0Q08CeRzt7M49w== 2024-09-18T00:57:05.389 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.y] 2024-09-18T00:57:05.389 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBCJepm08/pBhAAU+m4Qhr0k2Q3jjSdkPKP/Q== 2024-09-18T00:57:05.389 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.0] 2024-09-18T00:57:05.389 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBDJepmgARdFRAANYKMN+ZfR4AizHV0iyQ7zg== 2024-09-18T00:57:05.390 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.1] 2024-09-18T00:57:05.390 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBHJepmw1QIBBAAqq/q1IWTmb1Om8Vp88yCPQ== 2024-09-18T00:57:05.390 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.2] 2024-09-18T00:57:05.390 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBKJepmTWsvJBAAbPy2iYBpQwz9oaOUFymasQ== 2024-09-18T00:57:05.390 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.3] 2024-09-18T00:57:05.390 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBOJepmj9g/IxAAGr7gL5N5eX7/igYCgw+wYA== 2024-09-18T00:57:05.390 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.x] 2024-09-18T00:57:05.390 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBCJepmi/ylCRAA/EjXoM9tQOTDs3kB5874IQ== 2024-09-18T00:57:05.390 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.4] 2024-09-18T00:57:05.391 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBTJepmpSZ+AxAAMjfdfk7oHQfAQzp3HaXwdw== 2024-09-18T00:57:05.391 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.5] 2024-09-18T00:57:05.391 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBWJepmrLheIRAAw/ufOe0USo+dPFpOoWQ8HQ== 2024-09-18T00:57:05.391 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.6] 2024-09-18T00:57:05.391 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBaJepmVGSUBBAAY4ebUovOG6jHsfpVY7TSvg== 2024-09-18T00:57:05.391 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.7] 2024-09-18T00:57:05.391 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBdJepmCStTIhAAAZuWJrwvmzc2W0CliTTScw== 2024-09-18T00:57:05.392 INFO:teuthology.orchestra.run.smithi195.stdout:[client.0] 2024-09-18T00:57:05.392 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBCJepmF5t7DBAAOOrUAlQDk0egIiZJCQJmfA== 2024-09-18T00:57:05.392 INFO:teuthology.orchestra.run.smithi195.stdout:[client.1] 2024-09-18T00:57:05.392 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBCJepmvfM8DxAAbqiPFrGD0Q08CeRzt7M49w== 2024-09-18T00:57:05.393 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T00:57:05.427 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T00:57:05.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:05.481 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:05.525 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:05.571 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:05.613 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:05.659 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:05.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:05.744 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:05.790 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T00:57:05.827 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T00:57:05.872 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:05.911 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:05.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:05.958 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:06.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:06.044 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:06.129 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:06.132 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:57:06.219 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T00:57:06.221 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T00:57:06.265 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T00:57:06.307 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T00:57:06.352 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-18T00:57:06.352 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-18T00:57:06.364 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T00:57:06.451 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-18T00:57:06.461 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-18T00:57:06.520 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T00:57:06.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-18T00:57:06.619 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-18T00:57:06.633 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T00:57:06.728 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-18T00:57:06.740 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:57:06.742 DEBUG:teuthology.orchestra.run.smithi195:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:57:06.786 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-18T00:57:06.786 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-18T00:57:06.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-18T00:57:06.791 INFO:tasks.ceph.mon.a:Started 2024-09-18T00:57:06.791 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T00:57:06.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T00:57:06.796 INFO:tasks.ceph.mon.c:Started 2024-09-18T00:57:06.796 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-18T00:57:06.796 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-18T00:57:06.831 INFO:tasks.ceph.mon.b:Started 2024-09-18T00:57:06.831 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-18T00:57:06.831 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-18T00:57:06.832 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-18T00:57:06.834 INFO:tasks.ceph.mgr.y:Started 2024-09-18T00:57:06.834 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-18T00:57:06.834 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-18T00:57:06.836 INFO:tasks.ceph.mgr.x:Started 2024-09-18T00:57:06.836 INFO:tasks.ceph:Setting crush tunables to default 2024-09-18T00:57:06.836 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd crush tunables default 2024-09-18T00:57:07.334 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-09-18T00:57:07.359 INFO:tasks.ceph:check_enable_crimson: False 2024-09-18T00:57:07.359 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-18T00:57:07.359 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:57:07.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-18T00:57:07.372 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:57:07.372 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-18T00:57:07.387 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:07.382+0000 7f7314646180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T00:57:07.427 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:57:07.427 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-18T00:57:07.428 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:07.422+0000 7fe6b5339180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T00:57:07.439 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-18T00:57:07.439 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-18T00:57:07.492 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:57:07.492 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-18T00:57:07.506 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:57:07.506 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-18T00:57:07.562 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:57:07.562 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-18T00:57:07.617 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-18T00:57:07.617 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-18T00:57:07.673 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 3f359ce5-6f6e-4161-8554-8ac028347274 0 2024-09-18T00:57:07.921 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:07.914+0000 7f7314646180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T00:57:07.969 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:07.962+0000 7fe6b5339180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T00:57:08.051 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-18T00:57:08.046+0000 7f8da0c5d640 0 --2- 172.21.15.195:0/1485984297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d840052f0 0x7f8d9c1490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-18T00:57:08.165 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:08.158+0000 7f7314646180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T00:57:08.172 INFO:teuthology.orchestra.run.smithi195.stdout:0 2024-09-18T00:57:08.190 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 48687611-1556-4b6d-86af-7872494a91d7 1 2024-09-18T00:57:08.199 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:08.194+0000 7fe6b5339180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T00:57:08.391 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-09-18T00:57:08.407 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 9e6b1f90-ce9b-46d4-92fe-d457f7e415d5 2 2024-09-18T00:57:08.611 INFO:teuthology.orchestra.run.smithi195.stdout:2 2024-09-18T00:57:08.628 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5 3 2024-09-18T00:57:08.764 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:08.758+0000 7f7314646180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T00:57:08.790 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:08.786+0000 7fe6b5339180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T00:57:08.832 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:08.826+0000 7f7314646180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T00:57:08.848 INFO:teuthology.orchestra.run.smithi195.stdout:3 2024-09-18T00:57:08.848 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:08.842+0000 7fe6b5339180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T00:57:08.864 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 802ccbeb-7351-4b38-832a-695117c7e493 4 2024-09-18T00:57:08.887 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:08.882+0000 7f7314646180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T00:57:08.901 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:08.894+0000 7fe6b5339180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T00:57:08.967 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:08.962+0000 7f7314646180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T00:57:08.982 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:08.978+0000 7fe6b5339180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T00:57:09.021 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:09.014+0000 7f7314646180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T00:57:09.036 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:09.030+0000 7fe6b5339180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T00:57:09.064 INFO:teuthology.orchestra.run.smithi195.stdout:4 2024-09-18T00:57:09.081 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new d958b3da-efa1-4d86-b81d-e78874441437 5 2024-09-18T00:57:09.100 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:09.094+0000 7f7314646180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T00:57:09.117 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:09.110+0000 7fe6b5339180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T00:57:09.155 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:09.150+0000 7f7314646180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T00:57:09.172 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:09.166+0000 7fe6b5339180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T00:57:09.299 INFO:teuthology.orchestra.run.smithi195.stdout:5 2024-09-18T00:57:09.314 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:09.310+0000 7f7314646180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T00:57:09.316 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 51b3cea8-c686-4fd3-b3f4-ba936cf26b64 6 2024-09-18T00:57:09.330 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:09.326+0000 7fe6b5339180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T00:57:09.377 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:09.370+0000 7f7314646180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T00:57:09.394 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:09.390+0000 7fe6b5339180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T00:57:09.512 INFO:teuthology.orchestra.run.smithi195.stdout:6 2024-09-18T00:57:09.528 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new b02ead1c-837a-4b07-9ad1-4e903ed5bc4e 7 2024-09-18T00:57:09.762 INFO:teuthology.orchestra.run.smithi195.stdout:7 2024-09-18T00:57:09.786 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T00:57:09.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T00:57:09.788 INFO:tasks.ceph.osd.0:Started 2024-09-18T00:57:09.788 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T00:57:09.789 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T00:57:09.790 INFO:tasks.ceph.osd.1:Started 2024-09-18T00:57:09.790 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-18T00:57:09.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-18T00:57:09.792 INFO:tasks.ceph.osd.2:Started 2024-09-18T00:57:09.793 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-18T00:57:09.793 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-18T00:57:09.794 INFO:tasks.ceph.osd.3:Started 2024-09-18T00:57:09.795 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-18T00:57:09.795 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-18T00:57:09.797 INFO:tasks.ceph.osd.4:Started 2024-09-18T00:57:09.797 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-18T00:57:09.797 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-18T00:57:09.798 INFO:tasks.ceph.osd.5:Started 2024-09-18T00:57:09.799 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-18T00:57:09.799 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-18T00:57:09.800 INFO:tasks.ceph.osd.6:Started 2024-09-18T00:57:09.801 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-18T00:57:09.801 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-18T00:57:09.802 INFO:tasks.ceph.osd.7:Started 2024-09-18T00:57:09.803 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T00:57:09.994 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:57:09.995 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":10,"fsid":"82a96d76-ba83-49e2-a985-84604bd599ba","created":"2024-09-18T00:57:07.138732+0000","modified":"2024-09-18T00:57:09.756380+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T00:57:09.756380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3f359ce5-6f6e-4161-8554-8ac028347274","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"48687611-1556-4b6d-86af-7872494a91d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9e6b1f90-ce9b-46d4-92fe-d457f7e415d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"802ccbeb-7351-4b38-832a-695117c7e493","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d958b3da-efa1-4d86-b81d-e78874441437","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"51b3cea8-c686-4fd3-b3f4-ba936cf26b64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b02ead1c-837a-4b07-9ad1-4e903ed5bc4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:57:10.010 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-18T00:57:10.010 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-18T00:57:10.027 INFO:tasks.ceph.mgr.x.smithi195.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-18T00:57:10.027 INFO:tasks.ceph.mgr.x.smithi195.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-18T00:57:10.027 INFO:tasks.ceph.mgr.x.smithi195.stderr: from numpy import show_config as show_numpy_config 2024-09-18T00:57:10.069 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:10.062+0000 7f7314646180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-18T00:57:10.104 INFO:tasks.ceph.mgr.y.smithi042.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-18T00:57:10.104 INFO:tasks.ceph.mgr.y.smithi042.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-18T00:57:10.104 INFO:tasks.ceph.mgr.y.smithi042.stderr: from numpy import show_config as show_numpy_config 2024-09-18T00:57:10.165 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:10.158+0000 7fe6b5339180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-18T00:57:10.218 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:10.214+0000 7f7314646180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T00:57:10.338 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:10.334+0000 7fe6b5339180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T00:57:10.366 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:10.362+0000 7f7314646180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T00:57:10.429 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:10.422+0000 7f7314646180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T00:57:10.517 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:10.510+0000 7fe6b5339180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T00:57:10.563 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:10.558+0000 7f7314646180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T00:57:10.588 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:10.582+0000 7fe6b5339180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T00:57:10.739 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:10.734+0000 7fe6b5339180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T00:57:10.815 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T00:57:11.000 INFO:teuthology.misc.health.smithi042.stdout: 2024-09-18T00:57:11.004 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":10,"fsid":"82a96d76-ba83-49e2-a985-84604bd599ba","created":"2024-09-18T00:57:07.138732+0000","modified":"2024-09-18T00:57:09.756380+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T00:57:09.756380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3f359ce5-6f6e-4161-8554-8ac028347274","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"48687611-1556-4b6d-86af-7872494a91d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9e6b1f90-ce9b-46d4-92fe-d457f7e415d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"802ccbeb-7351-4b38-832a-695117c7e493","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d958b3da-efa1-4d86-b81d-e78874441437","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"51b3cea8-c686-4fd3-b3f4-ba936cf26b64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b02ead1c-837a-4b07-9ad1-4e903ed5bc4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:57:11.015 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-18T00:57:11.163 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:11.158+0000 7f7314646180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T00:57:11.235 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:11.230+0000 7f77f8da47c0 -1 Falling back to public interface 2024-09-18T00:57:11.238 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:11.234+0000 7f7314646180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T00:57:11.388 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:11.382+0000 7fe6b5339180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T00:57:11.420 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:11.414+0000 7f7314646180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T00:57:11.435 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:11.430+0000 7f9e719487c0 -1 Falling back to public interface 2024-09-18T00:57:11.451 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:11.446+0000 7fb8c7dd27c0 -1 Falling back to public interface 2024-09-18T00:57:11.467 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:11.462+0000 7fe6b5339180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T00:57:11.494 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:11.490+0000 7f7314646180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T00:57:11.658 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:11.654+0000 7fe6b5339180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T00:57:11.676 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:11.670+0000 7f92bda987c0 -1 Falling back to public interface 2024-09-18T00:57:11.679 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:11.674+0000 7f833c7a27c0 -1 Falling back to public interface 2024-09-18T00:57:11.706 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:11.702+0000 7faf549277c0 -1 Falling back to public interface 2024-09-18T00:57:11.735 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:11.730+0000 7fe6b5339180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T00:57:11.786 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:11.782+0000 7f7314646180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T00:57:11.848 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:11.842+0000 7f7314646180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T00:57:11.891 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:11.886+0000 7fbc0e3a07c0 -1 Falling back to public interface 2024-09-18T00:57:11.970 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:11.966+0000 7ff28ea2b7c0 -1 Falling back to public interface 2024-09-18T00:57:12.049 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:12.042+0000 7fe6b5339180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T00:57:12.114 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-18T00:57:12.106+0000 7fe6b5339180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T00:57:12.888 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:12.882+0000 7f72bb180640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-18T00:57:12.889 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:57:12.882+0000 7f72bb180640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-18T00:57:15.846 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:15.842+0000 7f833c7a27c0 -1 osd.7 0 log_to_monitors true 2024-09-18T00:57:15.900 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:15.894+0000 7f77f8da47c0 -1 osd.0 0 log_to_monitors true 2024-09-18T00:57:15.972 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:15.966+0000 7f92bda987c0 -1 osd.1 0 log_to_monitors true 2024-09-18T00:57:16.044 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:16.038+0000 7fb8c7dd27c0 -1 osd.4 0 log_to_monitors true 2024-09-18T00:57:16.071 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:16.066+0000 7fbc0e3a07c0 -1 osd.3 0 log_to_monitors true 2024-09-18T00:57:16.448 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:16.442+0000 7f9e719487c0 -1 osd.6 0 log_to_monitors true 2024-09-18T00:57:16.531 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:16.526+0000 7faf549277c0 -1 osd.5 0 log_to_monitors true 2024-09-18T00:57:16.547 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:16.542+0000 7ff28ea2b7c0 -1 osd.2 0 log_to_monitors true 2024-09-18T00:57:17.752 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:17.746+0000 7f8337f28640 -1 osd.7 0 waiting for initial osdmap 2024-09-18T00:57:17.786 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:17.782+0000 7f833056f640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:57:17.823 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T00:57:18.219 INFO:teuthology.misc.health.smithi042.stdout: 2024-09-18T00:57:18.219 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":13,"fsid":"82a96d76-ba83-49e2-a985-84604bd599ba","created":"2024-09-18T00:57:07.138732+0000","modified":"2024-09-18T00:57:17.888559+0000","last_up_change":"2024-09-18T00:57:17.888559+0000","last_in_change":"2024-09-18T00:57:09.756380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3f359ce5-6f6e-4161-8554-8ac028347274","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"48687611-1556-4b6d-86af-7872494a91d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9e6b1f90-ce9b-46d4-92fe-d457f7e415d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"802ccbeb-7351-4b38-832a-695117c7e493","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d958b3da-efa1-4d86-b81d-e78874441437","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"51b3cea8-c686-4fd3-b3f4-ba936cf26b64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b02ead1c-837a-4b07-9ad1-4e903ed5bc4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6817","nonce":2993352359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6819","nonce":2993352359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6823","nonce":2993352359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6821","nonce":2993352359}]},"public_addr":"172.21.15.195:6817/2993352359","cluster_addr":"172.21.15.195:6819/2993352359","heartbeat_back_addr":"172.21.15.195:6823/2993352359","heartbeat_front_addr":"172.21.15.195:6821/2993352359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:57:18.245 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-18T00:57:19.217 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:19.210+0000 7f9e6d0ce640 -1 osd.6 0 waiting for initial osdmap 2024-09-18T00:57:19.239 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:19.234+0000 7fb8c3558640 -1 osd.4 0 waiting for initial osdmap 2024-09-18T00:57:19.246 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:19.242+0000 7f9e65715640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:57:19.256 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:19.250+0000 7faf510e2640 -1 osd.5 0 waiting for initial osdmap 2024-09-18T00:57:19.270 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:19.266+0000 7fb8bbb9f640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:57:19.286 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:19.282+0000 7faf486f4640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:57:19.673 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:19.666+0000 7ff28b1e6640 -1 osd.2 0 waiting for initial osdmap 2024-09-18T00:57:19.703 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:19.698+0000 7ff2827f8640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:57:19.713 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:19.706+0000 7f92ba253640 -1 osd.1 0 waiting for initial osdmap 2024-09-18T00:57:19.739 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:19.734+0000 7f77f555f640 -1 osd.0 0 waiting for initial osdmap 2024-09-18T00:57:19.742 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:19.738+0000 7f92b1865640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:57:19.782 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:19.778+0000 7f77ecb71640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:57:19.793 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:19.786+0000 7fbc0ab5b640 -1 osd.3 0 waiting for initial osdmap 2024-09-18T00:57:19.822 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:19.818+0000 7fbc0216d640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:57:25.052 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T00:57:25.362 INFO:teuthology.misc.health.smithi042.stdout: 2024-09-18T00:57:25.363 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":19,"fsid":"82a96d76-ba83-49e2-a985-84604bd599ba","created":"2024-09-18T00:57:07.138732+0000","modified":"2024-09-18T00:57:24.281548+0000","last_up_change":"2024-09-18T00:57:20.258605+0000","last_in_change":"2024-09-18T00:57:09.756380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:57:21.962521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f359ce5-6f6e-4161-8554-8ac028347274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6801","nonce":199763443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6803","nonce":199763443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6807","nonce":199763443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6805","nonce":199763443}]},"public_addr":"172.21.15.42:6801/199763443","cluster_addr":"172.21.15.42:6803/199763443","heartbeat_back_addr":"172.21.15.42:6807/199763443","heartbeat_front_addr":"172.21.15.42:6805/199763443","state":["exists","up"]},{"osd":1,"uuid":"48687611-1556-4b6d-86af-7872494a91d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6809","nonce":345149800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6811","nonce":345149800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6815","nonce":345149800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6813","nonce":345149800}]},"public_addr":"172.21.15.42:6809/345149800","cluster_addr":"172.21.15.42:6811/345149800","heartbeat_back_addr":"172.21.15.42:6815/345149800","heartbeat_front_addr":"172.21.15.42:6813/345149800","state":["exists","up"]},{"osd":2,"uuid":"9e6b1f90-ce9b-46d4-92fe-d457f7e415d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6825","nonce":431239931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6827","nonce":431239931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6831","nonce":431239931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6829","nonce":431239931}]},"public_addr":"172.21.15.42:6825/431239931","cluster_addr":"172.21.15.42:6827/431239931","heartbeat_back_addr":"172.21.15.42:6831/431239931","heartbeat_front_addr":"172.21.15.42:6829/431239931","state":["exists","up"]},{"osd":3,"uuid":"dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6817","nonce":1775474180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6819","nonce":1775474180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6823","nonce":1775474180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6821","nonce":1775474180}]},"public_addr":"172.21.15.42:6817/1775474180","cluster_addr":"172.21.15.42:6819/1775474180","heartbeat_back_addr":"172.21.15.42:6823/1775474180","heartbeat_front_addr":"172.21.15.42:6821/1775474180","state":["exists","up"]},{"osd":4,"uuid":"802ccbeb-7351-4b38-832a-695117c7e493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6809","nonce":4198401207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6811","nonce":4198401207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6815","nonce":4198401207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6813","nonce":4198401207}]},"public_addr":"172.21.15.195:6809/4198401207","cluster_addr":"172.21.15.195:6811/4198401207","heartbeat_back_addr":"172.21.15.195:6815/4198401207","heartbeat_front_addr":"172.21.15.195:6813/4198401207","state":["exists","up"]},{"osd":5,"uuid":"d958b3da-efa1-4d86-b81d-e78874441437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6825","nonce":2599825823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6827","nonce":2599825823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6831","nonce":2599825823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6829","nonce":2599825823}]},"public_addr":"172.21.15.195:6825/2599825823","cluster_addr":"172.21.15.195:6827/2599825823","heartbeat_back_addr":"172.21.15.195:6831/2599825823","heartbeat_front_addr":"172.21.15.195:6829/2599825823","state":["exists","up"]},{"osd":6,"uuid":"51b3cea8-c686-4fd3-b3f4-ba936cf26b64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6801","nonce":3550656451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6803","nonce":3550656451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6807","nonce":3550656451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6805","nonce":3550656451}]},"public_addr":"172.21.15.195:6801/3550656451","cluster_addr":"172.21.15.195:6803/3550656451","heartbeat_back_addr":"172.21.15.195:6807/3550656451","heartbeat_front_addr":"172.21.15.195:6805/3550656451","state":["exists","up"]},{"osd":7,"uuid":"b02ead1c-837a-4b07-9ad1-4e903ed5bc4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6817","nonce":2993352359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6819","nonce":2993352359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6823","nonce":2993352359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6821","nonce":2993352359}]},"public_addr":"172.21.15.195:6817/2993352359","cluster_addr":"172.21.15.195:6819/2993352359","heartbeat_back_addr":"172.21.15.195:6823/2993352359","heartbeat_front_addr":"172.21.15.195:6821/2993352359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.896477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.941756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.560180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.077394+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.996264+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.511686+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.411051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.888092+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:57:25.381 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-18T00:57:25.381 INFO:tasks.ceph:Creating RBD pool 2024-09-18T00:57:25.381 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-18T00:57:26.301 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'rbd' created 2024-09-18T00:57:26.329 DEBUG:teuthology.orchestra.run.smithi042:> rbd --cluster ceph pool init rbd 2024-09-18T00:57:29.351 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-18T00:57:29.352 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-18T00:57:29.352 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-18T00:57:29.352 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-18T00:57:29.353 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-18T00:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2705509411},{"type":"v1","addr":"172.21.15.195:6833","nonce":2705509411}]},"active_addr":"172.21.15.195:6833/2705509411","active_change":"2024-09-18T00:57:11.869557+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":2937934590}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":2512818645}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":1257350385}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":1552573219}]}]} 2024-09-18T00:57:29.732 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-18T00:57:29.732 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T00:57:29.732 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T00:57:30.033 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:57:30.034 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"fsid":"82a96d76-ba83-49e2-a985-84604bd599ba","created":"2024-09-18T00:57:07.138732+0000","modified":"2024-09-18T00:57:29.319849+0000","last_up_change":"2024-09-18T00:57:20.258605+0000","last_in_change":"2024-09-18T00:57:09.756380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:57:21.962521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T00:57:25.721733+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f359ce5-6f6e-4161-8554-8ac028347274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6801","nonce":199763443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6803","nonce":199763443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6807","nonce":199763443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6805","nonce":199763443}]},"public_addr":"172.21.15.42:6801/199763443","cluster_addr":"172.21.15.42:6803/199763443","heartbeat_back_addr":"172.21.15.42:6807/199763443","heartbeat_front_addr":"172.21.15.42:6805/199763443","state":["exists","up"]},{"osd":1,"uuid":"48687611-1556-4b6d-86af-7872494a91d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6809","nonce":345149800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6811","nonce":345149800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6815","nonce":345149800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6813","nonce":345149800}]},"public_addr":"172.21.15.42:6809/345149800","cluster_addr":"172.21.15.42:6811/345149800","heartbeat_back_addr":"172.21.15.42:6815/345149800","heartbeat_front_addr":"172.21.15.42:6813/345149800","state":["exists","up"]},{"osd":2,"uuid":"9e6b1f90-ce9b-46d4-92fe-d457f7e415d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6825","nonce":431239931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6827","nonce":431239931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6831","nonce":431239931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6829","nonce":431239931}]},"public_addr":"172.21.15.42:6825/431239931","cluster_addr":"172.21.15.42:6827/431239931","heartbeat_back_addr":"172.21.15.42:6831/431239931","heartbeat_front_addr":"172.21.15.42:6829/431239931","state":["exists","up"]},{"osd":3,"uuid":"dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6817","nonce":1775474180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6819","nonce":1775474180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6823","nonce":1775474180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6821","nonce":1775474180}]},"public_addr":"172.21.15.42:6817/1775474180","cluster_addr":"172.21.15.42:6819/1775474180","heartbeat_back_addr":"172.21.15.42:6823/1775474180","heartbeat_front_addr":"172.21.15.42:6821/1775474180","state":["exists","up"]},{"osd":4,"uuid":"802ccbeb-7351-4b38-832a-695117c7e493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6809","nonce":4198401207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6811","nonce":4198401207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6815","nonce":4198401207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6813","nonce":4198401207}]},"public_addr":"172.21.15.195:6809/4198401207","cluster_addr":"172.21.15.195:6811/4198401207","heartbeat_back_addr":"172.21.15.195:6815/4198401207","heartbeat_front_addr":"172.21.15.195:6813/4198401207","state":["exists","up"]},{"osd":5,"uuid":"d958b3da-efa1-4d86-b81d-e78874441437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6825","nonce":2599825823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6827","nonce":2599825823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6831","nonce":2599825823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6829","nonce":2599825823}]},"public_addr":"172.21.15.195:6825/2599825823","cluster_addr":"172.21.15.195:6827/2599825823","heartbeat_back_addr":"172.21.15.195:6831/2599825823","heartbeat_front_addr":"172.21.15.195:6829/2599825823","state":["exists","up"]},{"osd":6,"uuid":"51b3cea8-c686-4fd3-b3f4-ba936cf26b64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6801","nonce":3550656451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6803","nonce":3550656451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6807","nonce":3550656451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6805","nonce":3550656451}]},"public_addr":"172.21.15.195:6801/3550656451","cluster_addr":"172.21.15.195:6803/3550656451","heartbeat_back_addr":"172.21.15.195:6807/3550656451","heartbeat_front_addr":"172.21.15.195:6805/3550656451","state":["exists","up"]},{"osd":7,"uuid":"b02ead1c-837a-4b07-9ad1-4e903ed5bc4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6817","nonce":2993352359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6819","nonce":2993352359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6823","nonce":2993352359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6821","nonce":2993352359}]},"public_addr":"172.21.15.195:6817/2993352359","cluster_addr":"172.21.15.195:6819/2993352359","heartbeat_back_addr":"172.21.15.195:6823/2993352359","heartbeat_front_addr":"172.21.15.195:6821/2993352359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.896477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.941756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.560180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.077394+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.996264+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.511686+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.411051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.888092+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:57:30.052 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T00:57:30.053 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T00:57:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:57:30.355 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"fsid":"82a96d76-ba83-49e2-a985-84604bd599ba","created":"2024-09-18T00:57:07.138732+0000","modified":"2024-09-18T00:57:29.319849+0000","last_up_change":"2024-09-18T00:57:20.258605+0000","last_in_change":"2024-09-18T00:57:09.756380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:57:21.962521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T00:57:25.721733+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f359ce5-6f6e-4161-8554-8ac028347274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6801","nonce":199763443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6803","nonce":199763443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6807","nonce":199763443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6805","nonce":199763443}]},"public_addr":"172.21.15.42:6801/199763443","cluster_addr":"172.21.15.42:6803/199763443","heartbeat_back_addr":"172.21.15.42:6807/199763443","heartbeat_front_addr":"172.21.15.42:6805/199763443","state":["exists","up"]},{"osd":1,"uuid":"48687611-1556-4b6d-86af-7872494a91d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6809","nonce":345149800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6811","nonce":345149800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6815","nonce":345149800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6813","nonce":345149800}]},"public_addr":"172.21.15.42:6809/345149800","cluster_addr":"172.21.15.42:6811/345149800","heartbeat_back_addr":"172.21.15.42:6815/345149800","heartbeat_front_addr":"172.21.15.42:6813/345149800","state":["exists","up"]},{"osd":2,"uuid":"9e6b1f90-ce9b-46d4-92fe-d457f7e415d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6825","nonce":431239931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6827","nonce":431239931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6831","nonce":431239931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6829","nonce":431239931}]},"public_addr":"172.21.15.42:6825/431239931","cluster_addr":"172.21.15.42:6827/431239931","heartbeat_back_addr":"172.21.15.42:6831/431239931","heartbeat_front_addr":"172.21.15.42:6829/431239931","state":["exists","up"]},{"osd":3,"uuid":"dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6817","nonce":1775474180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6819","nonce":1775474180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6823","nonce":1775474180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6821","nonce":1775474180}]},"public_addr":"172.21.15.42:6817/1775474180","cluster_addr":"172.21.15.42:6819/1775474180","heartbeat_back_addr":"172.21.15.42:6823/1775474180","heartbeat_front_addr":"172.21.15.42:6821/1775474180","state":["exists","up"]},{"osd":4,"uuid":"802ccbeb-7351-4b38-832a-695117c7e493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6809","nonce":4198401207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6811","nonce":4198401207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6815","nonce":4198401207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6813","nonce":4198401207}]},"public_addr":"172.21.15.195:6809/4198401207","cluster_addr":"172.21.15.195:6811/4198401207","heartbeat_back_addr":"172.21.15.195:6815/4198401207","heartbeat_front_addr":"172.21.15.195:6813/4198401207","state":["exists","up"]},{"osd":5,"uuid":"d958b3da-efa1-4d86-b81d-e78874441437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6825","nonce":2599825823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6827","nonce":2599825823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6831","nonce":2599825823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6829","nonce":2599825823}]},"public_addr":"172.21.15.195:6825/2599825823","cluster_addr":"172.21.15.195:6827/2599825823","heartbeat_back_addr":"172.21.15.195:6831/2599825823","heartbeat_front_addr":"172.21.15.195:6829/2599825823","state":["exists","up"]},{"osd":6,"uuid":"51b3cea8-c686-4fd3-b3f4-ba936cf26b64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6801","nonce":3550656451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6803","nonce":3550656451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6807","nonce":3550656451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6805","nonce":3550656451}]},"public_addr":"172.21.15.195:6801/3550656451","cluster_addr":"172.21.15.195:6803/3550656451","heartbeat_back_addr":"172.21.15.195:6807/3550656451","heartbeat_front_addr":"172.21.15.195:6805/3550656451","state":["exists","up"]},{"osd":7,"uuid":"b02ead1c-837a-4b07-9ad1-4e903ed5bc4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6817","nonce":2993352359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6819","nonce":2993352359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6823","nonce":2993352359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6821","nonce":2993352359}]},"public_addr":"172.21.15.195:6817/2993352359","cluster_addr":"172.21.15.195:6819/2993352359","heartbeat_back_addr":"172.21.15.195:6823/2993352359","heartbeat_front_addr":"172.21.15.195:6821/2993352359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.896477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.941756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.560180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.077394+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.996264+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.511686+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.411051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.888092+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:57:30.373 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T00:57:30.373 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T00:57:30.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T00:57:30.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T00:57:30.375 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T00:57:30.375 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T00:57:30.375 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T00:57:30.376 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T00:57:30.633 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-18T00:57:30.634 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T00:57:30.635 INFO:teuthology.orchestra.run.smithi042.stdout:64424509444 2024-09-18T00:57:30.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T00:57:30.643 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-18T00:57:30.643 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T00:57:30.648 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-18T00:57:30.648 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T00:57:30.650 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-18T00:57:30.651 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T00:57:30.654 INFO:teuthology.orchestra.run.smithi042.stdout:64424509444 2024-09-18T00:57:30.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T00:57:30.656 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-18T00:57:30.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T00:57:30.665 INFO:teuthology.orchestra.run.smithi042.stdout:55834574853 2024-09-18T00:57:30.666 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T00:57:31.161 INFO:teuthology.orchestra.run.smithi042.stdout:64424509444 2024-09-18T00:57:31.177 INFO:teuthology.orchestra.run.smithi042.stdout:64424509443 2024-09-18T00:57:31.182 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-18T00:57:31.185 INFO:teuthology.orchestra.run.smithi042.stdout:64424509443 2024-09-18T00:57:31.186 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-18T00:57:31.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.1 2024-09-18T00:57:31.209 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-18T00:57:31.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-09-18T00:57:31.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-09-18T00:57:31.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.6 2024-09-18T00:57:31.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.5 2024-09-18T00:57:31.225 INFO:teuthology.orchestra.run.smithi042.stdout:64424509444 2024-09-18T00:57:31.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2024-09-18T00:57:31.235 INFO:teuthology.orchestra.run.smithi042.stdout:55834574852 2024-09-18T00:57:31.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.0 2024-09-18T00:57:31.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-09-18T00:57:32.192 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T00:57:32.213 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T00:57:32.214 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T00:57:32.217 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T00:57:32.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T00:57:32.230 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T00:57:32.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T00:57:32.254 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T00:57:32.742 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-18T00:57:32.750 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-18T00:57:32.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.1 2024-09-18T00:57:32.767 DEBUG:teuthology.parallel:result is None 2024-09-18T00:57:32.779 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.6 2024-09-18T00:57:32.780 DEBUG:teuthology.parallel:result is None 2024-09-18T00:57:32.789 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-18T00:57:32.791 INFO:teuthology.orchestra.run.smithi042.stdout:64424509447 2024-09-18T00:57:32.795 INFO:teuthology.orchestra.run.smithi042.stdout:64424509447 2024-09-18T00:57:32.800 INFO:teuthology.orchestra.run.smithi042.stdout:55834574853 2024-09-18T00:57:32.801 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-18T00:57:32.802 INFO:teuthology.orchestra.run.smithi042.stdout:64424509447 2024-09-18T00:57:32.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.5 2024-09-18T00:57:32.828 DEBUG:teuthology.parallel:result is None 2024-09-18T00:57:32.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.4 2024-09-18T00:57:32.830 DEBUG:teuthology.parallel:result is None 2024-09-18T00:57:32.831 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.0 2024-09-18T00:57:32.831 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.2 2024-09-18T00:57:32.831 DEBUG:teuthology.parallel:result is None 2024-09-18T00:57:32.831 DEBUG:teuthology.parallel:result is None 2024-09-18T00:57:32.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.3 2024-09-18T00:57:32.834 DEBUG:teuthology.parallel:result is None 2024-09-18T00:57:32.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-09-18T00:57:32.836 DEBUG:teuthology.parallel:result is None 2024-09-18T00:57:32.836 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T00:57:32.836 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:57:33.159 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:57:33.159 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:57:33.176 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-18T00:57:31.883764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256684,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754718036,"statfs":{"total":773094113280,"available":772831268864,"internally_reserved":0,"allocated":3063808,"data_stored":1814286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.582451"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.328163+0000","last_change":"2024-09-18T00:57:29.328163+0000","last_active":"2024-09-18T00:57:29.328163+0000","last_peered":"2024-09-18T00:57:29.328163+0000","last_clean":"2024-09-18T00:57:29.328163+0000","last_became_active":"2024-09-18T00:57:27.311231+0000","last_became_peered":"2024-09-18T00:57:27.311231+0000","last_unstale":"2024-09-18T00:57:29.328163+0000","last_undegraded":"2024-09-18T00:57:29.328163+0000","last_fullsized":"2024-09-18T00:57:29.328163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:57.340012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020990940000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327841+0000","last_change":"2024-09-18T00:57:29.327841+0000","last_active":"2024-09-18T00:57:29.327841+0000","last_peered":"2024-09-18T00:57:29.327841+0000","last_clean":"2024-09-18T00:57:29.327841+0000","last_became_active":"2024-09-18T00:57:27.315417+0000","last_became_peered":"2024-09-18T00:57:27.315417+0000","last_unstale":"2024-09-18T00:57:29.327841+0000","last_undegraded":"2024-09-18T00:57:29.327841+0000","last_fullsized":"2024-09-18T00:57:29.327841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:54.335334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327995+0000","last_change":"2024-09-18T00:57:29.327995+0000","last_active":"2024-09-18T00:57:29.327995+0000","last_peered":"2024-09-18T00:57:29.327995+0000","last_clean":"2024-09-18T00:57:29.327995+0000","last_became_active":"2024-09-18T00:57:27.316238+0000","last_became_peered":"2024-09-18T00:57:27.316238+0000","last_unstale":"2024-09-18T00:57:29.327995+0000","last_undegraded":"2024-09-18T00:57:29.327995+0000","last_fullsized":"2024-09-18T00:57:29.327995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019441149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.339600+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:57:29.339600+0000","last_peered":"2024-09-18T00:57:29.339600+0000","last_clean":"2024-09-18T00:57:29.339600+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:57:29.339600+0000","last_undegraded":"2024-09-18T00:57:29.339600+0000","last_fullsized":"2024-09-18T00:57:29.339600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.334219+0000","last_change":"2024-09-18T00:57:29.327436+0000","last_active":"2024-09-18T00:57:29.334219+0000","last_peered":"2024-09-18T00:57:29.334219+0000","last_clean":"2024-09-18T00:57:29.334219+0000","last_became_active":"2024-09-18T00:57:27.410600+0000","last_became_peered":"2024-09-18T00:57:27.410600+0000","last_unstale":"2024-09-18T00:57:29.334219+0000","last_undegraded":"2024-09-18T00:57:29.334219+0000","last_fullsized":"2024-09-18T00:57:29.334219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:59.380173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.653878+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:57:29.653878+0000","last_peered":"2024-09-18T00:57:29.653878+0000","last_clean":"2024-09-18T00:57:29.653878+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:57:29.653878+0000","last_undegraded":"2024-09-18T00:57:29.653878+0000","last_fullsized":"2024-09-18T00:57:29.653878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327276+0000","last_change":"2024-09-18T00:57:29.327276+0000","last_active":"2024-09-18T00:57:29.327276+0000","last_peered":"2024-09-18T00:57:29.327276+0000","last_clean":"2024-09-18T00:57:29.327276+0000","last_became_active":"2024-09-18T00:57:27.316217+0000","last_became_peered":"2024-09-18T00:57:27.316217+0000","last_unstale":"2024-09-18T00:57:29.327276+0000","last_undegraded":"2024-09-18T00:57:29.327276+0000","last_fullsized":"2024-09-18T00:57:29.327276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:38.026720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012774469999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.328733+0000","last_change":"2024-09-18T00:57:29.328733+0000","last_active":"2024-09-18T00:57:29.328733+0000","last_peered":"2024-09-18T00:57:29.328733+0000","last_clean":"2024-09-18T00:57:29.328733+0000","last_became_active":"2024-09-18T00:57:27.410396+0000","last_became_peered":"2024-09-18T00:57:27.410396+0000","last_unstale":"2024-09-18T00:57:29.328733+0000","last_undegraded":"2024-09-18T00:57:29.328733+0000","last_fullsized":"2024-09-18T00:57:29.328733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:27.438117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327217+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:57:29.327217+0000","last_peered":"2024-09-18T00:57:29.327217+0000","last_clean":"2024-09-18T00:57:29.327217+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:57:29.327217+0000","last_undegraded":"2024-09-18T00:57:29.327217+0000","last_fullsized":"2024-09-18T00:57:29.327217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":79208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":79208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35584,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336256,"statfs":{"total":96636764160,"available":96600326144,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:57:33.177 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:57:33.466 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:57:33.467 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:57:33.484 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-18T00:57:31.883764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256684,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754718036,"statfs":{"total":773094113280,"available":772831268864,"internally_reserved":0,"allocated":3063808,"data_stored":1814286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.582451"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.328163+0000","last_change":"2024-09-18T00:57:29.328163+0000","last_active":"2024-09-18T00:57:29.328163+0000","last_peered":"2024-09-18T00:57:29.328163+0000","last_clean":"2024-09-18T00:57:29.328163+0000","last_became_active":"2024-09-18T00:57:27.311231+0000","last_became_peered":"2024-09-18T00:57:27.311231+0000","last_unstale":"2024-09-18T00:57:29.328163+0000","last_undegraded":"2024-09-18T00:57:29.328163+0000","last_fullsized":"2024-09-18T00:57:29.328163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:57.340012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020990940000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327841+0000","last_change":"2024-09-18T00:57:29.327841+0000","last_active":"2024-09-18T00:57:29.327841+0000","last_peered":"2024-09-18T00:57:29.327841+0000","last_clean":"2024-09-18T00:57:29.327841+0000","last_became_active":"2024-09-18T00:57:27.315417+0000","last_became_peered":"2024-09-18T00:57:27.315417+0000","last_unstale":"2024-09-18T00:57:29.327841+0000","last_undegraded":"2024-09-18T00:57:29.327841+0000","last_fullsized":"2024-09-18T00:57:29.327841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:54.335334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327995+0000","last_change":"2024-09-18T00:57:29.327995+0000","last_active":"2024-09-18T00:57:29.327995+0000","last_peered":"2024-09-18T00:57:29.327995+0000","last_clean":"2024-09-18T00:57:29.327995+0000","last_became_active":"2024-09-18T00:57:27.316238+0000","last_became_peered":"2024-09-18T00:57:27.316238+0000","last_unstale":"2024-09-18T00:57:29.327995+0000","last_undegraded":"2024-09-18T00:57:29.327995+0000","last_fullsized":"2024-09-18T00:57:29.327995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019441149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.339600+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:57:29.339600+0000","last_peered":"2024-09-18T00:57:29.339600+0000","last_clean":"2024-09-18T00:57:29.339600+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:57:29.339600+0000","last_undegraded":"2024-09-18T00:57:29.339600+0000","last_fullsized":"2024-09-18T00:57:29.339600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.334219+0000","last_change":"2024-09-18T00:57:29.327436+0000","last_active":"2024-09-18T00:57:29.334219+0000","last_peered":"2024-09-18T00:57:29.334219+0000","last_clean":"2024-09-18T00:57:29.334219+0000","last_became_active":"2024-09-18T00:57:27.410600+0000","last_became_peered":"2024-09-18T00:57:27.410600+0000","last_unstale":"2024-09-18T00:57:29.334219+0000","last_undegraded":"2024-09-18T00:57:29.334219+0000","last_fullsized":"2024-09-18T00:57:29.334219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:59.380173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.653878+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:57:29.653878+0000","last_peered":"2024-09-18T00:57:29.653878+0000","last_clean":"2024-09-18T00:57:29.653878+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:57:29.653878+0000","last_undegraded":"2024-09-18T00:57:29.653878+0000","last_fullsized":"2024-09-18T00:57:29.653878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327276+0000","last_change":"2024-09-18T00:57:29.327276+0000","last_active":"2024-09-18T00:57:29.327276+0000","last_peered":"2024-09-18T00:57:29.327276+0000","last_clean":"2024-09-18T00:57:29.327276+0000","last_became_active":"2024-09-18T00:57:27.316217+0000","last_became_peered":"2024-09-18T00:57:27.316217+0000","last_unstale":"2024-09-18T00:57:29.327276+0000","last_undegraded":"2024-09-18T00:57:29.327276+0000","last_fullsized":"2024-09-18T00:57:29.327276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:38.026720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012774469999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.328733+0000","last_change":"2024-09-18T00:57:29.328733+0000","last_active":"2024-09-18T00:57:29.328733+0000","last_peered":"2024-09-18T00:57:29.328733+0000","last_clean":"2024-09-18T00:57:29.328733+0000","last_became_active":"2024-09-18T00:57:27.410396+0000","last_became_peered":"2024-09-18T00:57:27.410396+0000","last_unstale":"2024-09-18T00:57:29.328733+0000","last_undegraded":"2024-09-18T00:57:29.328733+0000","last_fullsized":"2024-09-18T00:57:29.328733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:27.438117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327217+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:57:29.327217+0000","last_peered":"2024-09-18T00:57:29.327217+0000","last_clean":"2024-09-18T00:57:29.327217+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:57:29.327217+0000","last_undegraded":"2024-09-18T00:57:29.327217+0000","last_fullsized":"2024-09-18T00:57:29.327217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":79208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":79208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35584,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336256,"statfs":{"total":96636764160,"available":96600326144,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:57:33.485 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T00:57:33.485 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-18T00:57:33.485 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-18T00:57:33.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-18T00:57:33.804 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:57:33.805 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-18T00:57:33.822 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-18T00:57:33.823 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-18T00:57:33.834 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-18T00:57:33.834 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}} 2024-09-18T00:57:33.834 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-18T00:57:33.834 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-18T00:57:34.140 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-18T00:57:34.140 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T00:57:34.140 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:57:34.422 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:57:34.422 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:57:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-18T00:57:33.884378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256684,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754718036,"statfs":{"total":773094113280,"available":772831268864,"internally_reserved":0,"allocated":3063808,"data_stored":1814286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.580642"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.328163+0000","last_change":"2024-09-18T00:57:29.328163+0000","last_active":"2024-09-18T00:57:29.328163+0000","last_peered":"2024-09-18T00:57:29.328163+0000","last_clean":"2024-09-18T00:57:29.328163+0000","last_became_active":"2024-09-18T00:57:27.311231+0000","last_became_peered":"2024-09-18T00:57:27.311231+0000","last_unstale":"2024-09-18T00:57:29.328163+0000","last_undegraded":"2024-09-18T00:57:29.328163+0000","last_fullsized":"2024-09-18T00:57:29.328163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:57.340012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020990940000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327841+0000","last_change":"2024-09-18T00:57:29.327841+0000","last_active":"2024-09-18T00:57:29.327841+0000","last_peered":"2024-09-18T00:57:29.327841+0000","last_clean":"2024-09-18T00:57:29.327841+0000","last_became_active":"2024-09-18T00:57:27.315417+0000","last_became_peered":"2024-09-18T00:57:27.315417+0000","last_unstale":"2024-09-18T00:57:29.327841+0000","last_undegraded":"2024-09-18T00:57:29.327841+0000","last_fullsized":"2024-09-18T00:57:29.327841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:54.335334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327995+0000","last_change":"2024-09-18T00:57:29.327995+0000","last_active":"2024-09-18T00:57:29.327995+0000","last_peered":"2024-09-18T00:57:29.327995+0000","last_clean":"2024-09-18T00:57:29.327995+0000","last_became_active":"2024-09-18T00:57:27.316238+0000","last_became_peered":"2024-09-18T00:57:27.316238+0000","last_unstale":"2024-09-18T00:57:29.327995+0000","last_undegraded":"2024-09-18T00:57:29.327995+0000","last_fullsized":"2024-09-18T00:57:29.327995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019441149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.339600+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:57:29.339600+0000","last_peered":"2024-09-18T00:57:29.339600+0000","last_clean":"2024-09-18T00:57:29.339600+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:57:29.339600+0000","last_undegraded":"2024-09-18T00:57:29.339600+0000","last_fullsized":"2024-09-18T00:57:29.339600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.334219+0000","last_change":"2024-09-18T00:57:29.327436+0000","last_active":"2024-09-18T00:57:29.334219+0000","last_peered":"2024-09-18T00:57:29.334219+0000","last_clean":"2024-09-18T00:57:29.334219+0000","last_became_active":"2024-09-18T00:57:27.410600+0000","last_became_peered":"2024-09-18T00:57:27.410600+0000","last_unstale":"2024-09-18T00:57:29.334219+0000","last_undegraded":"2024-09-18T00:57:29.334219+0000","last_fullsized":"2024-09-18T00:57:29.334219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:59.380173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.653878+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:57:29.653878+0000","last_peered":"2024-09-18T00:57:29.653878+0000","last_clean":"2024-09-18T00:57:29.653878+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:57:29.653878+0000","last_undegraded":"2024-09-18T00:57:29.653878+0000","last_fullsized":"2024-09-18T00:57:29.653878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327276+0000","last_change":"2024-09-18T00:57:29.327276+0000","last_active":"2024-09-18T00:57:29.327276+0000","last_peered":"2024-09-18T00:57:29.327276+0000","last_clean":"2024-09-18T00:57:29.327276+0000","last_became_active":"2024-09-18T00:57:27.316217+0000","last_became_peered":"2024-09-18T00:57:27.316217+0000","last_unstale":"2024-09-18T00:57:29.327276+0000","last_undegraded":"2024-09-18T00:57:29.327276+0000","last_fullsized":"2024-09-18T00:57:29.327276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:38.026720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012774469999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.328733+0000","last_change":"2024-09-18T00:57:29.328733+0000","last_active":"2024-09-18T00:57:29.328733+0000","last_peered":"2024-09-18T00:57:29.328733+0000","last_clean":"2024-09-18T00:57:29.328733+0000","last_became_active":"2024-09-18T00:57:27.410396+0000","last_became_peered":"2024-09-18T00:57:27.410396+0000","last_unstale":"2024-09-18T00:57:29.328733+0000","last_undegraded":"2024-09-18T00:57:29.328733+0000","last_fullsized":"2024-09-18T00:57:29.328733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:27.438117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327217+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:57:29.327217+0000","last_peered":"2024-09-18T00:57:29.327217+0000","last_clean":"2024-09-18T00:57:29.327217+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:57:29.327217+0000","last_undegraded":"2024-09-18T00:57:29.327217+0000","last_fullsized":"2024-09-18T00:57:29.327217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":79208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":79208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35584,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336256,"statfs":{"total":96636764160,"available":96600326144,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:57:34.440 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:57:34.719 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:57:34.720 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:57:34.736 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-18T00:57:33.884378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256684,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754718036,"statfs":{"total":773094113280,"available":772831268864,"internally_reserved":0,"allocated":3063808,"data_stored":1814286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.580642"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.328163+0000","last_change":"2024-09-18T00:57:29.328163+0000","last_active":"2024-09-18T00:57:29.328163+0000","last_peered":"2024-09-18T00:57:29.328163+0000","last_clean":"2024-09-18T00:57:29.328163+0000","last_became_active":"2024-09-18T00:57:27.311231+0000","last_became_peered":"2024-09-18T00:57:27.311231+0000","last_unstale":"2024-09-18T00:57:29.328163+0000","last_undegraded":"2024-09-18T00:57:29.328163+0000","last_fullsized":"2024-09-18T00:57:29.328163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:57.340012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020990940000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327841+0000","last_change":"2024-09-18T00:57:29.327841+0000","last_active":"2024-09-18T00:57:29.327841+0000","last_peered":"2024-09-18T00:57:29.327841+0000","last_clean":"2024-09-18T00:57:29.327841+0000","last_became_active":"2024-09-18T00:57:27.315417+0000","last_became_peered":"2024-09-18T00:57:27.315417+0000","last_unstale":"2024-09-18T00:57:29.327841+0000","last_undegraded":"2024-09-18T00:57:29.327841+0000","last_fullsized":"2024-09-18T00:57:29.327841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:54.335334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327995+0000","last_change":"2024-09-18T00:57:29.327995+0000","last_active":"2024-09-18T00:57:29.327995+0000","last_peered":"2024-09-18T00:57:29.327995+0000","last_clean":"2024-09-18T00:57:29.327995+0000","last_became_active":"2024-09-18T00:57:27.316238+0000","last_became_peered":"2024-09-18T00:57:27.316238+0000","last_unstale":"2024-09-18T00:57:29.327995+0000","last_undegraded":"2024-09-18T00:57:29.327995+0000","last_fullsized":"2024-09-18T00:57:29.327995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019441149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.339600+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:57:29.339600+0000","last_peered":"2024-09-18T00:57:29.339600+0000","last_clean":"2024-09-18T00:57:29.339600+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:57:29.339600+0000","last_undegraded":"2024-09-18T00:57:29.339600+0000","last_fullsized":"2024-09-18T00:57:29.339600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.334219+0000","last_change":"2024-09-18T00:57:29.327436+0000","last_active":"2024-09-18T00:57:29.334219+0000","last_peered":"2024-09-18T00:57:29.334219+0000","last_clean":"2024-09-18T00:57:29.334219+0000","last_became_active":"2024-09-18T00:57:27.410600+0000","last_became_peered":"2024-09-18T00:57:27.410600+0000","last_unstale":"2024-09-18T00:57:29.334219+0000","last_undegraded":"2024-09-18T00:57:29.334219+0000","last_fullsized":"2024-09-18T00:57:29.334219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:59.380173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.653878+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:57:29.653878+0000","last_peered":"2024-09-18T00:57:29.653878+0000","last_clean":"2024-09-18T00:57:29.653878+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:57:29.653878+0000","last_undegraded":"2024-09-18T00:57:29.653878+0000","last_fullsized":"2024-09-18T00:57:29.653878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327276+0000","last_change":"2024-09-18T00:57:29.327276+0000","last_active":"2024-09-18T00:57:29.327276+0000","last_peered":"2024-09-18T00:57:29.327276+0000","last_clean":"2024-09-18T00:57:29.327276+0000","last_became_active":"2024-09-18T00:57:27.316217+0000","last_became_peered":"2024-09-18T00:57:27.316217+0000","last_unstale":"2024-09-18T00:57:29.327276+0000","last_undegraded":"2024-09-18T00:57:29.327276+0000","last_fullsized":"2024-09-18T00:57:29.327276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:38.026720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012774469999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.328733+0000","last_change":"2024-09-18T00:57:29.328733+0000","last_active":"2024-09-18T00:57:29.328733+0000","last_peered":"2024-09-18T00:57:29.328733+0000","last_clean":"2024-09-18T00:57:29.328733+0000","last_became_active":"2024-09-18T00:57:27.410396+0000","last_became_peered":"2024-09-18T00:57:27.410396+0000","last_unstale":"2024-09-18T00:57:29.328733+0000","last_undegraded":"2024-09-18T00:57:29.328733+0000","last_fullsized":"2024-09-18T00:57:29.328733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:27.438117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T00:57:29.327217+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:57:29.327217+0000","last_peered":"2024-09-18T00:57:29.327217+0000","last_clean":"2024-09-18T00:57:29.327217+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:57:29.327217+0000","last_undegraded":"2024-09-18T00:57:29.327217+0000","last_fullsized":"2024-09-18T00:57:29.327217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":79208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":79208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35584,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336256,"statfs":{"total":96636764160,"available":96600326144,"internally_reserved":0,"allocated":233472,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:57:34.737 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T00:57:34.737 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-18T00:57:35.017 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 23 2024-09-18T00:57:35.017 INFO:teuthology.orchestra.run.smithi042.stdout:fsid 82a96d76-ba83-49e2-a985-84604bd599ba 2024-09-18T00:57:35.017 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-18T00:57:07.138732+0000 2024-09-18T00:57:35.017 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-18T00:57:29.319849+0000 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:crush_version 5 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:full_ratio 0.95 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:backfillfull_ratio 0.9 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:nearfull_ratio 0.85 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:require_min_compat_client luminous 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:min_compat_client jewel 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:require_osd_release squid 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:stretch_mode_enabled false 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 23 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.98 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout: removed_snaps_queue [2~1] 2024-09-18T00:57:35.018 INFO:teuthology.orchestra.run.smithi042.stdout:max_osd 8 2024-09-18T00:57:35.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6800/199763443,v1:172.21.15.42:6801/199763443] [v2:172.21.15.42:6802/199763443,v1:172.21.15.42:6803/199763443] exists,up 3f359ce5-6f6e-4161-8554-8ac028347274 2024-09-18T00:57:35.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 up in weight 1 up_from 15 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6808/345149800,v1:172.21.15.42:6809/345149800] [v2:172.21.15.42:6810/345149800,v1:172.21.15.42:6811/345149800] exists,up 48687611-1556-4b6d-86af-7872494a91d7 2024-09-18T00:57:35.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6824/431239931,v1:172.21.15.42:6825/431239931] [v2:172.21.15.42:6826/431239931,v1:172.21.15.42:6827/431239931] exists,up 9e6b1f90-ce9b-46d4-92fe-d457f7e415d5 2024-09-18T00:57:35.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6816/1775474180,v1:172.21.15.42:6817/1775474180] [v2:172.21.15.42:6818/1775474180,v1:172.21.15.42:6819/1775474180] exists,up dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5 2024-09-18T00:57:35.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6808/4198401207,v1:172.21.15.195:6809/4198401207] [v2:172.21.15.195:6810/4198401207,v1:172.21.15.195:6811/4198401207] exists,up 802ccbeb-7351-4b38-832a-695117c7e493 2024-09-18T00:57:35.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6824/2599825823,v1:172.21.15.195:6825/2599825823] [v2:172.21.15.195:6826/2599825823,v1:172.21.15.195:6827/2599825823] exists,up d958b3da-efa1-4d86-b81d-e78874441437 2024-09-18T00:57:35.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6800/3550656451,v1:172.21.15.195:6801/3550656451] [v2:172.21.15.195:6802/3550656451,v1:172.21.15.195:6803/3550656451] exists,up 51b3cea8-c686-4fd3-b3f4-ba936cf26b64 2024-09-18T00:57:35.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6816/2993352359,v1:172.21.15.195:6817/2993352359] [v2:172.21.15.195:6818/2993352359,v1:172.21.15.195:6819/2993352359] exists,up b02ead1c-837a-4b07-9ad1-4e903ed5bc4e 2024-09-18T00:57:35.035 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6800/199763443,v1:172.21.15.42:6801/199763443] [v2:172.21.15.42:6802/199763443,v1:172.21.15.42:6803/199763443] exists,up 3f359ce5-6f6e-4161-8554-8ac028347274', 'osd.1 up in weight 1 up_from 15 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6808/345149800,v1:172.21.15.42:6809/345149800] [v2:172.21.15.42:6810/345149800,v1:172.21.15.42:6811/345149800] exists,up 48687611-1556-4b6d-86af-7872494a91d7', 'osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6824/431239931,v1:172.21.15.42:6825/431239931] [v2:172.21.15.42:6826/431239931,v1:172.21.15.42:6827/431239931] exists,up 9e6b1f90-ce9b-46d4-92fe-d457f7e415d5', 'osd.3 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6816/1775474180,v1:172.21.15.42:6817/1775474180] [v2:172.21.15.42:6818/1775474180,v1:172.21.15.42:6819/1775474180] exists,up dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5', 'osd.4 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6808/4198401207,v1:172.21.15.195:6809/4198401207] [v2:172.21.15.195:6810/4198401207,v1:172.21.15.195:6811/4198401207] exists,up 802ccbeb-7351-4b38-832a-695117c7e493', 'osd.5 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6824/2599825823,v1:172.21.15.195:6825/2599825823] [v2:172.21.15.195:6826/2599825823,v1:172.21.15.195:6827/2599825823] exists,up d958b3da-efa1-4d86-b81d-e78874441437', 'osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6800/3550656451,v1:172.21.15.195:6801/3550656451] [v2:172.21.15.195:6802/3550656451,v1:172.21.15.195:6803/3550656451] exists,up 51b3cea8-c686-4fd3-b3f4-ba936cf26b64', 'osd.7 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6816/2993352359,v1:172.21.15.195:6817/2993352359] [v2:172.21.15.195:6818/2993352359,v1:172.21.15.195:6819/2993352359] exists,up b02ead1c-837a-4b07-9ad1-4e903ed5bc4e'] 2024-09-18T00:57:35.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-18T00:57:35.152 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:35.152 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mon.a", 2024-09-18T00:57:35.152 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster": "ceph", 2024-09-18T00:57:35.152 INFO:teuthology.orchestra.run.smithi042.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-18T00:57:35.153 INFO:teuthology.orchestra.run.smithi042.stdout: "admin_socket_mode": "", 2024-09-18T00:57:35.153 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-18T00:57:35.153 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_client_required": "cephx, none", 2024-09-18T00:57:35.153 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_cluster_required": "cephx", 2024-09-18T00:57:35.153 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_debug": "true", 2024-09-18T00:57:35.153 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-18T00:57:35.153 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-18T00:57:35.153 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_service_required": "cephx", 2024-09-18T00:57:35.153 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-18T00:57:35.153 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_supported": "cephx", 2024-09-18T00:57:35.153 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio": "true", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_poll_ms": "250", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_reap_max": "16", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_async_discard": "false", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_block_size": "4096", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio": "false", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_inflight_ios": "false", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_enable_discard": "false", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_flock_retry": "3", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-18T00:57:35.154 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_inject_crash": "0", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring": "false", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring_hipri": "false", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_type": "", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_alloc_size": "1048576", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_allocator": "hybrid", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_buffered_io": "true", 2024-09-18T00:57:35.155 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_for_zeros": "false", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_compact_log_sync": "false", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_max_log_runway": "4194304", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_max_prefetch": "1048576", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_min_flush_size": "524288", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_min_log_runway": "1048576", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_replay_recovery": "false", 2024-09-18T00:57:35.156 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_sync_write": "false", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_allocation_from_file": "true", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_allocator": "hybrid", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bdev_label_multi": "true", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-18T00:57:35.157 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_create": "true", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_create": "false", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_path": "", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_size": "0", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_path": "", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_preallocate_file": "false", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_size": "107374182400", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_create": "false", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_path": "", 2024-09-18T00:57:35.158 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_size": "100663296", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs": "true", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_autotune": "true", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-18T00:57:35.159 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size": "0", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_type": "2q", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_clone_cow": "true", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-18T00:57:35.160 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_mode": "none", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_csum_type": "crc32c", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_freelist": "false", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-18T00:57:35.161 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_small_allocations": "0", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_default_buffered_read": "true", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_default_buffered_write": "false", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-18T00:57:35.162 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fail_eio": "false", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-18T00:57:35.163 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mount": "false", 2024-09-18T00:57:35.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-18T00:57:35.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_umount": "false", 2024-09-18T00:57:35.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-18T00:57:35.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-18T00:57:35.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-18T00:57:35.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-18T00:57:35.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-18T00:57:35.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-18T00:57:35.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-18T00:57:35.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-18T00:57:35.164 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_ignore_data_csum": "false", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_op_age": "5.000000", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_alloc_size": "0", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size": "0", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_deferred_txc": "32", 2024-09-18T00:57:35.165 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size": "0", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_nid_prealloc": "1024", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_retry_disk_reads": "3", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_cf": "true", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-18T00:57:35.166 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_mem": "512", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_tracing": "false", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-18T00:57:35.167 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_zero_block_detection": "false", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-18T00:57:35.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_cluster_require_signatures": "false", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_cluster_require_version": "2", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_require_signatures": "false", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_require_version": "2", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_service_require_signatures": "false", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_service_require_version": "2", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_sign_messages": "true", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "chdir": "", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client_acl_type": "", 2024-09-18T00:57:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client_asio_thread_count": "2", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_cache_mid": "0.750000", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_cache_size": "16384", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_caps_release_delay": "5", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_check_pool_perm": "true", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_force_sync_read": "false", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_getattr_caps": "false", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_inject_features": "", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_inject_tick_delay": "0", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_die_on_failed_remount": "false", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_dirsize_rbytes": "true", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_force_lazyio": "false", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_fs": "", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-18T00:57:35.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_inject_release_failure": "false", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_max_inline_size": "4096", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mds_namespace": "", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_metadata": "", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_gid": "-1", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_timeout": "300", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_uid": "-1", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mountpoint": "/", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_notify_timeout": "10", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc": "true", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_dirty": "104857600", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_objects": "1000", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_size": "209715200", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_target_dirty": "8388608", 2024-09-18T00:57:35.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_permissions": "true", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_quota": "true", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_quota_df": "true", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_max_bytes": "0", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_max_periods": "4", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_min": "131072", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_reconnect_stale": "false", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_shutdown_timeout": "30", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_snapdir": ".snap", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_tick_interval": "1", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_trace": "", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_try_dentry_invalidate": "false", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_use_faked_inos": "false", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "client_use_random_mds": "false", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog": "false", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-18T00:57:35.172 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog_port": "12201", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_monitors": "default=true", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog": "false", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog_level": "info", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "-", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_network": "", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_network_interface": "", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_isal": "false", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_level": "5", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_winsize": "-15", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zstd_level": "1", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-18T00:57:35.173 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_alien_op_num_threads": "6", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_osd_stat_interval": "0", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_seastar_cpu_cores": "", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_seastar_num_threads": "0", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location": "", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location_hook": "", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location_hook_timeout": "10", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "daemonize": "false", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "daos_pool": "tank", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-18T00:57:35.174 INFO:teuthology.orchestra.run.smithi042.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_alienstore": "0/5", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_allow_any_pool_priority": "false", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asok": "1/5", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asok_assert_abort": "false", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asserts_on_shutdown": "true", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_auth": "20/20", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bdev": "1/3", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bluefs": "1/5", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bluestore": "1/5", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_buffer": "0/1", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_ceph_exporter": "1/5", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_cephfs_mirror": "0/5", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_cephsqlite": "0/5", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_client": "0/5", 2024-09-18T00:57:35.175 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_compressor": "1/5", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_context": "0/1", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_crimson_interrupt": "0/5", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_crush": "1/1", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_crypto": "1/5", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_cyanstore": "0/5", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_deliberately_leak_memory": "false", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_disable_randomized_ping": "false", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_dpdk": "1/5", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_eventtrace": "1/5", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_filer": "0/1", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_filestore": "1/3", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_finisher": "1/1", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_fuse": "1/5", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_heartbeat_testing_span": "0", 2024-09-18T00:57:35.176 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_heartbeatmap": "1/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_javaclient": "1/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_journal": "1/3", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_journaler": "0/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_kstore": "1/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_lockdep": "0/1", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mclock": "1/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds": "1/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_balancer": "1/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_locker": "1/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_log": "1/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_log_expire": "1/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_migrator": "1/5", 2024-09-18T00:57:35.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_quiesce": "3/5", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_memstore": "1/5", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mgr": "2/5", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mgrc": "1/5", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mon": "20/20", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_monc": "0/10", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_ms": "1/1", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_none": "0/5", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objclass": "0/5", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objectcacher": "0/5", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objecter": "0/1", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_optracker": "0/5", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_osd": "1/5", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_paxos": "20/20", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_perfcounter": "1/5", 2024-09-18T00:57:35.178 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_prioritycache": "1/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rados": "0/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd": "0/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_mirror": "0/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_pwl": "0/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_replay": "0/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_refs": "0/0", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_reserver": "1/1", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw": "1/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_access": "1/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_datacache": "1/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_dbstore": "1/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_flight": "1/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_sync": "1/5", 2024-09-18T00:57:35.179 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rocksdb": "4/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_backref": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_cache": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_cleaner": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_device": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_epm": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_journal": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_lba": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_odata": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_omap": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_onode": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_t": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_tm": "0/5", 2024-09-18T00:57:35.180 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_striper": "0/1", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_test": "0/5", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_throttle": "1/1", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_timer": "0/1", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_tp": "0/5", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_trace": "1/5", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "device_failure_prediction_mode": "none", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_graylog": "false", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_journald": "false", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_stderr": "true", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_syslog": "false", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event_tracing": "false", 2024-09-18T00:57:35.181 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_addr": "0.0.0.0", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_http_port": "9926", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_prio_limit": "5", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_sort_metrics": "true", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_stats_period": "5", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "fake_statfs_for_testing": "0", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "fatal_signal_handlers": "true", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "filer_max_purge_ops": "10", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "filer_max_truncate_ops": "128", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_apply_finisher_threads": "1", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_blackhole": "false", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_btrfs_clone_range": "true", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_btrfs_snap": "true", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_caller_concurrency": "10", 2024-09-18T00:57:35.182 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_collect_device_partition_information": "true", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_commit_timeout": "600.000000", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_inject_read_err": "false", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_omap_check": "false", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_verify_split": "false", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_dump_file": "", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fadvise": "true", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fail_eio": "true", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fd_cache_shards": "16", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fd_cache_size": "128", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fiemap": "false", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fiemap_threshold": "4096", 2024-09-18T00:57:35.183 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_inject_stall": "0", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_parallel": "false", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_trailing": "false", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_writeahead": "false", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_kill_at": "0", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs": "0", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-18T00:57:35.184 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size": "0", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_merge_threshold": "-10", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_odsync_write": "false", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_backend": "rocksdb", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_backend_path": "", 2024-09-18T00:57:35.185 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_thread_timeout": "60", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_threads": "2", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_punch_hole": "false", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-18T00:57:35.186 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_ops": "50", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_seek_data_hole": "false", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_sloppy_crc": "false", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_splice": "false", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_split_multiple": "2", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_split_rand_factor": "20", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_update_to": "1000", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-18T00:57:35.187 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_enable": "true", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_xfs_extsize": "false", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_zfs_snap": "false", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "fio_dir": "/tmp/fio", 2024-09-18T00:57:35.188 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "82a96d76-ba83-49e2-a985-84604bd599ba", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_allow_other": "true", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_atomic_o_trunc": "true", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_big_writes": "true", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_debug": "false", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_default_permissions": "false", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_disable_pagecache": "false", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_max_write": "0", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_multithreaded": "true", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_require_active_mds": "true", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_set_user_groups": "true", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_move": "true", 2024-09-18T00:57:35.189 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_read": "true", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_write": "true", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_use_invalidate_cb": "true", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "gss_target_name": "ceph", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_file": "", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_inject_failure": "0", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_interval": "5", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "host": "smithi042", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-18T00:57:35.190 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "inject_early_sigterm": "false", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "jaeger_agent_port": "6799", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "jaeger_tracing_enable": "false", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_aio": "true", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_align_min_size": "65536", 2024-09-18T00:57:35.191 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_block_align": "true", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_block_size": "4096", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_dio": "true", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_discard": "false", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_force_aio": "false", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_ignore_corruption": "false", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_max_write_bytes": "10485760", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_max_write_entries": "100", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_replay_from": "0", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_write_header_frequency": "0", 2024-09-18T00:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_zero_on_create": "false", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_prefetch_periods": "10", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_prezero_periods": "5", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_write_head_interval": "15", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "keyfile": "", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_backend": "rocksdb", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_default_stripe_size": "65536", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_fsck_on_mount": "false", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_max_bytes": "67108864", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_max_ops": "512", 2024-09-18T00:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_nid_prealloc": "1024", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_onode_map_size": "1024", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_sync_submit_transaction": "false", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_sync_transaction": "false", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "librados_thread_count": "2", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "lockdep": "false", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "lockdep_force_backtrace": "false", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "log_coarse_timestamps": "true", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "log_flush_on_exit": "false", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "log_graylog_host": "127.0.0.1", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "log_graylog_port": "12201", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "log_max_new": "1000", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "log_max_recent": "10000", 2024-09-18T00:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: "log_stderr_prefix": "", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "log_stop_at_utilization": "0.970000", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": "true", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_graylog": "false", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_journald": "false", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_stderr": "false", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_syslog": "false", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "max_rotating_auth_attempts": "10", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_action_on_write_error": "1", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_alternate_name_max": "8192", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_asio_thread_count": "2", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_export_pin": "true", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_dirs": "true", 2024-09-18T00:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_interval": "5", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_interval": "10", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_max": "-1", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_max_until": "-1", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_merge_size": "50", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_midchunk": "0.300000", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_min_start": "0.200000", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_minchunk": "0.001000", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_mode": "0", 2024-09-18T00:57:35.196 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_need_max": "1.200000", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_need_min": "0.800000", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_overload_epochs": "2", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_bits": "3", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_size": "10000", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_target_decay": "10.000000", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_grace": "15.000000", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_interval": "4.000000", 2024-09-18T00:57:35.197 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_mid": "0.700000", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_quiesce_delay": "0", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_release_free_interval": "10", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_reservation": "0.050000", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_interval": "1", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_threshold": "262144", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-18T00:57:35.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_prealloc_inos": "1000", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_connect_bootstrapping": "false", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_damage_table_max_entries": "10000", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_auth_pins": "false", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_frag": "false", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_scatterstat": "false", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_subtrees": "false", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_decay_halflife": "5.000000", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_default_dir_hash": "2", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_defer_session_stale": "true", 2024-09-18T00:57:35.199 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_deny_all_reconnect": "false", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_keys_per_op": "16384", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_max_commit_size": "10", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_max_entries": "0", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_prefetch": "true", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_on_map": "false", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_early_reply": "true", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_enable_op_tracker": "true", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_enforce_unique_name": "true", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-18T00:57:35.200 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_random": "true", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_health_summarize_threshold": "10", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_health_dummy": "false", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_migrator_session_race": "false", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-18T00:57:35.201 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_join_fs": "", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_journal_format": "1", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_create_at": "0", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_export_at": "0", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_import_at": "0", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_at": "0", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_expire_at": "0", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_replay_at": "0", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_link_at": "0", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_mdstable_at": "0", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_openc_at": "0", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_rename_at": "0", 2024-09-18T00:57:35.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_shutdown_at": "0", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_event_large_threshold": "524288", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_events_per_segment": "1024", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_max_events": "-1", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_max_segments": "128", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_pause": "false", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_segment_size": "0", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_trim_threshold": "128", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_warn_factor": "2.000000", 2024-09-18T00:57:35.203 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_caps_per_client": "1048576", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_completed_flushes": "100000", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_completed_requests": "100000", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_export_size": "20971520", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_file_recover": "32", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_files": "64", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_ops": "8192", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_snaps_per_dir": "100", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_metrics_update_interval": "2", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_min_caps_per_client": "100", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_min_caps_working_set": "10000", 2024-09-18T00:57:35.204 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_numa_node": "-1", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_complaint_time": "30.000000", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_duration": "600", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_size": "20", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_slow_op_size": "20", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_log_threshold": "5", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_ping_grace": "15", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_ping_interval": "5", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_caps": "30000", 2024-09-18T00:57:35.205 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_warning_threshold": "262144", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_replay_interval": "1.000000", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_root_ino_gid": "0", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_root_ino_uid": "0", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-18T00:57:35.206 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_shutdown_check": "0", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_skip_ino": "0", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_max_uid": "4294967294", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_min_uid": "0", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_rstat": "false", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_standby_replay_damaged": "false", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_symlink_recovery": "true", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_thrash_exports": "0", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_thrash_fragments": "0", 2024-09-18T00:57:35.207 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_tick_interval": "5.000000", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_valgrind_exit": "false", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_verify_backtrace": "1", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_verify_scatter": "false", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_wipe_sessions": "false", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool_debug": "false", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_device_bytes": "1073741824", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_page_set": "false", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_page_size": "65536", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_bytes": "134217728", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_messages": "512", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-18T00:57:35.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_disabled_modules": "", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_enable_op_tracker": "true", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_max_pg_num_change": "128", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mds_bytes": "134217728", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mds_messages": "128", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mon_bytes": "134217728", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mon_messages": "128", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_history_duration": "600", 2024-09-18T00:57:35.209 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_history_size": "20", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_log_threshold": "5", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_osd_bytes": "536870912", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_osd_messages": "8192", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_pool": "true", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_standby_modules": "true", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_stats_period": "5", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_stats_threshold": "5", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_tick_period": "2", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_allow_pool_delete": "true", 2024-09-18T00:57:35.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_allow_pool_size_one": "true", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_auth_validate_all_caps": "true", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_bytes": "104857600", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_directed_command_retry": "5", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_parallel": "3", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_log_interval": "1.000000", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_ping_interval": "10.000000", 2024-09-18T00:57:35.211 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_target_rank": "-1", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_file_level": "debug", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_file": "true", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_journald": "false", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-18T00:57:35.212 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_bootstrap": "false", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_start": "false", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_trim": "true", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cpu_threads": "4", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_crush_min_required_version": "hammer", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_daemon_bytes": "419430400", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_avail_crit": "5", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_avail_warn": "5", 2024-09-18T00:57:35.213 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_size_warn": "16106127360", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_json": "false", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_transactions": "false", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_extra_checks": "false", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_reef": "false", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_squid": "false", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-18T00:57:35.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-18T00:57:35.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-18T00:57:35.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_down_mkfs_grace": "120", 2024-09-18T00:57:35.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_down_uptime_grace": "60", 2024-09-18T00:57:35.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_election_default_strategy": "1", 2024-09-18T00:57:35.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_election_timeout": "5.000000", 2024-09-18T00:57:35.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-18T00:57:35.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ping_divisor": "2", 2024-09-18T00:57:35.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-18T00:57:35.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_enable_op_tracker": "true", 2024-09-18T00:57:35.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_fake_pool_delete": "false", 2024-09-18T00:57:35.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_force_quorum_join": "false", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_globalid_prealloc": "10000", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_detail_to_clog": "false", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_log_update_period": "5", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_max_detail": "50", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog": "true", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog_interval": "600", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_host": "172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.195", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_host_override": "", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_initial_members": "", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-18T00:57:35.216 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease": "5.000000", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_full_interval": "50", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_max": "10000", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_max_summary": "50", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_log_epochs": "500", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_osd": "10000", 2024-09-18T00:57:35.217 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_pg_per_osd": "10000", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_pool_pg_num": "65536", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_force_trim_to": "0", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_skip_sanity": "false", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_memory_autotune": "true", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_memory_target": "2147483648", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_beacon_grace": "30", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_digest_period": "5", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_inactive_grace": "60", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-18T00:57:35.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_min_osdmap_epochs": "500", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_complaint_time": "30", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_duration": "600", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_size": "20", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_slow_op_size": "20", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_log_threshold": "5", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_in": "false", 2024-09-18T00:57:35.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_cache_size": "500", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_down_out_interval": "600", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_force_trim_to": "0", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-18T00:57:35.220 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_down_reporters": "2", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_report_timeout": "900", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-18T00:57:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_stuck_threshold": "60", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-18T00:57:35.222 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_probe_timeout": "2.000000", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_max_change": "0.050000", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_max_osds": "4", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_interval": "86400", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_max_keys": "100", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_timeout": "300", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_session_timeout": "300", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_smart_report_timeout": "5", 2024-09-18T00:57:35.223 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stat_smooth_intervals": "6", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_pool_min_size": "2", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_pool_size": "4", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_debug": "false", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_provider_kill_at": "0", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_requester_kill_at": "0", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_timeout": "60.000000", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_target_pg_per_osd": "100", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_tick_interval": "5", 2024-09-18T00:57:35.224 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_timecheck_interval": "300.000000", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_use_min_delay_socket": "false", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_older_version_delay": "604800", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_misplaced": "false", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_older_version": "true", 2024-09-18T00:57:35.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": "", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-18T00:57:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_tracing_enabled": "false", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_op_threads": "3", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_cm": "false", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_device_name": "", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_dscp": "96", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_local_gid": "", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_port_num": "1", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-18T00:57:35.227 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_sl": "3", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_support_srq": "true", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_type": "ib", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_reap_threshold": "5", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_before_connect": "false", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_ipv4": "true", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_ipv6": "false", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_msgr1": "true", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_msgr2": "true", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_port_max": "7568", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_port_min": "6800", 2024-09-18T00:57:35.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_retry_count": "3", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_retry_delay": "5", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_client": "false", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mds": "false", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mgr": "false", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mon": "false", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_osd": "false", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_client_mode": "crc secure", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_cluster_mode": "crc secure", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_cluster_type": "", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_compress_secure": "false", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_connection_idle_timeout": "900", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_connection_ready_timeout": "10", 2024-09-18T00:57:35.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_crc_data": "true", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_crc_header": "true", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_bad_msg": "false", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_bug": "true", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_old_message": "true", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_skipped_message": "false", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_coremask": "0xF", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_enable_tso": "true", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-18T00:57:35.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hugepages": "", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_lro": "true", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_memory_channel": "4", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_pmd": "", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_port_id": "0", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dump_on_send": "false", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_initial_backoff": "0.200000", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_max": "1.000000", 2024-09-18T00:57:35.231 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_type": "", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_network_congestion": "0", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_socket_failures": "5000", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_learn_addr_from_peer": "true", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_max_accept_failures": "4", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_max_backoff": "15.000000", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_client_mode": "secure crc", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_service_mode": "secure crc", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compress_min_size": "1024", 2024-09-18T00:57:35.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compress_mode": "none", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_pq_min_cost": "65536", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_public_type": "", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_service_mode": "crc secure", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_listen_backlog": "512", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_nodelay": "true", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_rcvbuf": "0", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_type": "async+posix", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "no_config_file": "false", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_completion_locks_per_session": "32", 2024-09-18T00:57:35.233 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inflight_ops": "1024", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_tick_interval": "5.000000", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_timeout": "10.000000", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "objectstore_blackhole": "false", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "openssl_engine_opts": "", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_delay_time": "5.000000", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_hist_halflife": "1000", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_max_low_ops": "2", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_max_ops": "4", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-18T00:57:35.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_slop": "0.020000", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_asio_thread_count": "2", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_async_recovery_min_cost": "100", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_scan_max": "512", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_scan_min": "64", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_degraded": "false", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_peering": "false", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_unfound": "true", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_beacon_report_interval": "300", 2024-09-18T00:57:35.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_duration": "30", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_max_block_size": "67108864", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_blkin_trace_all": "false", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_check_for_log_corruption": "false", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_update_on_start": "true", 2024-09-18T00:57:35.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_message_cap": "256", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_message_size_cap": "524288000", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_op_priority": "63", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_watch_timeout": "30", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_command_max_records": "256", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_compact_on_start": "false", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_update_on_start": "true", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_update_weight_set": "true", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-18T00:57:35.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_feed_pullee": "-1", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_misdirected_ops": "false", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_no_acting_change": "false", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_no_purge_strays": "false", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_op_order": "false", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-18T00:57:35.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_shutdown": "false", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_trim_objects": "false", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_snaps": "false", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_keys": "1024", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_stride": "524288", 2024-09-18T00:57:35.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_default_notify_timeout": "30", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep": "0.000000", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_discard_disconnected_ops": "true", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_enable_op_tracker": "true", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-18T00:57:35.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_info": "true", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown": "true", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_function_tracing": "false", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_grace": "20", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_interval": "6", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-18T00:57:35.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_peers": "10", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_size": "2000", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_stale": "600", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_max_size": "100000", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_min_size": "1000", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal_size": "5120", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_kill_backfill_at": "0", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_cache_size": "50", 2024-09-18T00:57:35.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_dedup": "true", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_message_max": "40", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_share_max_epochs": "40", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_attr_name_len": "100", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_attr_size": "0", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_backfills": "1", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_markdown_count": "5", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_markdown_period": "600", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_name_len": "2048", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_namespace_len": "256", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_size": "134217728", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_blocked_by": "16", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_log_entries": "10000", 2024-09-18T00:57:35.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pgls": "1024", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_push_cost": "8388608", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_push_objects": "10", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_scrubs": "3", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_trimming_pgs": "2", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_write_op_reply_len": "64", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_write_size": "90", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-18T00:57:35.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_profile": "balanced", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-18T00:57:35.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_base": "805306368", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_cache_min": "134217728", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target": "4294967296", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target_autotune": "false", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_min_pg_log_entries": "250", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_min_recovery_priority": "0", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_report_interval": "5", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_num_cache_shards": "32", 2024-09-18T00:57:35.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_num_op_tracker_shard": "32", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_auto_affinity": "true", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_node": "-1", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_prefer_iface": "true", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objecter_finishers": "1", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore": "bluestore", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore_fuse": "false", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore_tracing": "false", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_complaint_time": "30.000000", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_duration": "600", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_size": "20", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_slow_op_size": "20", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_log_threshold": "5", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards": "0", 2024-09-18T00:57:35.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards_hdd": "5", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards_ssd": "8", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_pq_min_cost": "65536", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_queue_cut_off": "high", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_thread_timeout": "15", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_open_classes_on_start": "true", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_os_flags": "0", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_peering_op_priority": "255", 2024-09-18T00:57:35.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_delete_cost": "1048576", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_delete_priority": "5", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_trim_max": "10000", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_trim_min": "100", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-18T00:57:35.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_crimson": "false", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flags": "0", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_min_size": "0", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pg_num": "32", 2024-09-18T00:57:35.250 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pgp_num": "0", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_read_ratio": "70", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_size": "2", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_type": "replicated", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_push_per_object_cost": "1000", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recover_clone_overlap": "true", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_cost": "20971520", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active": "0", 2024-09-18T00:57:35.251 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_single_start": "1", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_op_priority": "3", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_priority": "5", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep": "0.000000", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_repair_during_recovery": "false", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_requested_scrub_priority": "120", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-18T00:57:35.252 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_auto_repair": "false", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_begin_hour": "0", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_begin_week_day": "0", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_chunk_max": "25", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_chunk_min": "5", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_cost": "52428800", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_during_recovery": "false", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_end_hour": "0", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_end_week_day": "0", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_event_cost": "4096", 2024-09-18T00:57:35.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_invalid_stats": "true", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_max_preemptions": "5", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_priority": "5", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_sleep": "0.000000", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_skip_data_digest": "false", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_smart_report_timeout": "5", 2024-09-18T00:57:35.254 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_cost": "1048576", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_priority": "5", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_target_transaction_size": "30", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-18T00:57:35.255 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tracing": "false", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_use_stale_snap": "false", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osdc_blkin_trace_all": "false", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_kill_at": "0", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_max_join_drift": "10", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_min": "500", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_min_wait": "0.050000", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_propose_interval": "1.000000", 2024-09-18T00:57:35.256 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_max": "500", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_min": "250", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_stash_full_interval": "25", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_trim_max": "500", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_trim_min": "250", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "perf": "true", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "-", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrv": "", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "public_bind_addr": "-", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "public_network": "", 2024-09-18T00:57:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "public_network_interface": "", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "qat_compressor_busy_polling": "false", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "qat_compressor_enabled": "false", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "qat_compressor_session_max_number": "256", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_mon_op_timeout": "0", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_osd_op_timeout": "0", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_tracing": "false", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_atime_update_interval": "60", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_balance_parent_reads": "false", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_balance_snap_reads": "false", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blkin_trace_all": "false", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache": "true", 2024-09-18T00:57:35.258 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_policy": "writearound", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_size": "33554432", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_clone_copy_on_read": "false", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_compression_hint": "none", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_concurrent_management_ops": "10", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_clone_format": "auto", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_data_pool": "", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_features": "61", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_format": "2", 2024-09-18T00:57:35.259 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_map_options": "", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_order": "22", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_pool": "rbd", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_stripe_count": "0", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_stripe_unit": "0", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_enable_alloc_hint": "true", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_io_scheduler": "simple", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-18T00:57:35.260 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_order": "24", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_pool": "", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_splay_width": "4", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_localize_parent_reads": "false", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_localize_snap_reads": "false", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-18T00:57:35.261 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-18T00:57:35.262 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-18T00:57:35.263 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mtime_update_interval": "60", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_non_blocking_aio": "true", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_op_thread_timeout": "60", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_op_threads": "1", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_parent_cache_enabled": "false", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_plugins": "", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_burst": "0", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-18T00:57:35.264 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_limit": "0", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_exclude_ops": "0", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_burst": "0", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_limit": "0", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-18T00:57:35.265 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_read_from_replica_policy": "default", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_skip_partial_discard": "true", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_tracing": "false", 2024-09-18T00:57:35.266 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_validate_names": "true", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_validate_pool": "true", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "restapi_base_url": "", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "restapi_log_level": "", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_acl_grants_max_num": "100", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_admin_entry": "admin", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_backend_store": "rados", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_barbican_url": "", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_beast_enable_async": "true", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_counters_cache": "false", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-18T00:57:35.267 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_enabled": "true", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_expiry_interval": "900", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_lru_size": "25000", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_config_store": "rados", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_content_length_compat": "false", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_copy_obj_progress": "true", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-18T00:57:35.268 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cors_rules_max_num": "100", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cross_domain_policy": "", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_addr": "", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_password": "", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_username": "", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_require_ssl": "true", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-18T00:57:35.269 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-18T00:57:35.270 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_addr": "", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_auth": "token", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_namespace": "", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_prefix": "", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_token_file": "", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_buffersize": "524288", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_low_speed_time": "300", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-18T00:57:35.271 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d4n_port": "6379", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_changes_size": "1000", 2024-09-18T00:57:35.272 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_num_shards": "128", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_window": "30", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-18T00:57:35.273 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dns_name": "", 2024-09-18T00:57:35.274 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dns_s3website_name": "", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dynamic_resharding": "true", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_gc_threads": "true", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_lc_threads": "true", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_ops_log": "false", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_quota_threads": "true", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_static_website": "false", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_usage_log": "false", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enforce_swift_acls": "true", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_exit_timeout_secs": "120", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_expose_bucket": "false", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_extended_http_attrs": "", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_filter": "none", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-18T00:57:35.275 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_frontends": "beast port=7480", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_deferred": "50", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_objs": "32", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_processor_period": "3600", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-18T00:57:35.276 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_init_timeout": "300", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_kafka_connection_idle": "300", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_domain": "", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_password": "", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_password_path": "", 2024-09-18T00:57:35.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_project": "", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_tenant": "", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_token": "", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_token_path": "", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_user": "", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_api_version": "2", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_domain": "", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_password": "", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_project": "", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_user": "", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-18T00:57:35.278 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-18T00:57:35.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-18T00:57:35.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-18T00:57:35.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-18T00:57:35.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_url": "", 2024-09-18T00:57:35.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-18T00:57:35.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_debug_interval": "-1", 2024-09-18T00:57:35.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_lock_max_time": "90", 2024-09-18T00:57:35.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_objs": "32", 2024-09-18T00:57:35.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_rules": "1000", 2024-09-18T00:57:35.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_worker": "3", 2024-09-18T00:57:35.279 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_thread_delay": "0", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_dnattr": "uid", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_searchfilter": "", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_uri": "", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_http_headers": "", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-18T00:57:35.280 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_object_name_utc": "false", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attr_name_len": "0", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attr_size": "0", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_chunk_size": "4194304", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_listing_results": "1000", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_notify_retries": "10", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_put_param_size": "1048576", 2024-09-18T00:57:35.281 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_put_size": "5368709120", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_slo_entries": "1000", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_md_log_max_shards": "64", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_mp_lock_max_time": "600", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-18T00:57:35.282 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_max_gc": "300", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_num_async_rados_threads": "32", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_num_control_oids": "8", 2024-09-18T00:57:35.283 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_numa_node": "-1", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_chunk_size": "100", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_gc_interval": "600", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_thread_timeout": "600", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_tracing": "false", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_token": "", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_url": "", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_verify_ssl": "true", 2024-09-18T00:57:35.284 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_rados": "false", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_socket_path": "", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_push_interval": "2.000000", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_cache_lanes": "3", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-18T00:57:35.285 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_cache_partitions": "3", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_print_continue": "true", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_tracing": "false", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm": "", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm_id": "", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_region": "", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-18T00:57:35.286 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_request_uri": "", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_batch_size": "64", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_max_aio": "128", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_num_logs": "16", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_thread_interval": "600", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_resolve_cname": "false", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_run_sync_thread": "true", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-18T00:57:35.287 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_scheduler_type": "throttler", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_script_uri": "", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_service_provider_name": "", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_client_id": "", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_client_secret": "", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_entry": "sts", 2024-09-18T00:57:35.288 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_key": "sts", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_min_session_duration": "900", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_token_introspection_url": "", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_account_in_url": "false", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_auth_entry": "auth", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_auth_url": "", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_custom_header": "", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_need_stats": "true", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_tenant_name": "", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_token_expiration": "86400", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_url": "", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_url_prefix": "swift", 2024-09-18T00:57:35.289 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_lease_period": "120", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-18T00:57:35.290 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_thread_pool_size": "512", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_comment": "", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_createby": "", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_encoding": "", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_flag": "false", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_origin": "", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_tracker": "", 2024-09-18T00:57:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_trust_forwarded_https": "false", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_max_shards": "32", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_max_user_shards": "1", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_use_opa_authz": "false", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_counters_cache": "false", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_max_buckets": "1000", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_policies_max_num": "100", 2024-09-18T00:57:35.292 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_unique_email": "true", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_verify_ssl": "true", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone": "", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone_id": "", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup": "", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup_id": "", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-18T00:57:35.293 INFO:teuthology.orchestra.run.smithi042.stdout: "rgwlc_auto_session_clear": "true", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_block_size": "4096", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_size": "536870912", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-18T00:57:35.294 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_index_type": "binary_search", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_partition_filters": "false", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_perf": "false", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "run_dir": "/var/run/ceph", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_block_create": "true", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_cache_lru_size": "67108864", 2024-09-18T00:57:35.295 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_default_max_object_size": "16777216", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_device_size": "53687091200", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_full_integrity_check": "false", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_capacity": "16", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_main_device_type": "SSD", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-18T00:57:35.296 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_segment_size": "67108864", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "set_keepcaps": "false", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "setgroup": "", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "setuser": "", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "setuser_match_path": "", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "thp": "false", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "threadpool_default_timeout": "60", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout: "throttler_perf_counter": "true" 2024-09-18T00:57:35.297 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:35.300 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-18T00:57:35.445 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a: {} 2024-09-18T00:57:35.445 INFO:teuthology.orchestra.run.smithi042.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T00:57:35.457 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b: {} 2024-09-18T00:57:35.457 INFO:teuthology.orchestra.run.smithi042.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T00:57:35.467 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c: {} 2024-09-18T00:57:35.467 INFO:teuthology.orchestra.run.smithi042.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T00:57:35.483 DEBUG:teuthology.orchestra.run.smithi195:> type ceph-objectstore-tool 2024-09-18T00:57:35.488 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T00:57:35.488 DEBUG:teuthology.orchestra.run.smithi042:> type ceph-objectstore-tool 2024-09-18T00:57:35.492 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T00:57:35.493 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-18T00:57:35.503 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-18T00:57:35.503 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-18T00:57:35.503 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:57:35.503 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:57:35.503 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T00:57:35.504 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T00:57:35.504 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-18T00:57:35.504 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-18T00:57:35.505 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:57:35.505 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-18T00:57:35.505 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:57:35.505 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-18T00:57:35.506 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:57:35.506 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-18T00:57:35.535 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:35.530+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:35.554 INFO:teuthology.orchestra.run.smithi042.stdout:mapped writes DISABLED 2024-09-18T00:57:35.554 INFO:teuthology.orchestra.run.smithi042.stdout:mapped reads DISABLED 2024-09-18T00:57:35.554 INFO:teuthology.orchestra.run.smithi042.stdout:Seed set to 5976 2024-09-18T00:57:35.606 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:35.602+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:35.652 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:35.652 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:35.652 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:35.652 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:35.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:57:35.707 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:35.702+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:35.743 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:57:35.744 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:57:35.776 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:57:35.776 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:57:35.807 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:57:35.807 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:57:35.808 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:35.802+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:35.823 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:57:35.823 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:57:35.834 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:35.834 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:35.844 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:57:35.844 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:57:35.853 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:57:35.864 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:57:35.865 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:57:35.885 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:57:35.885 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:57:35.907 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:57:35.907 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:57:35.909 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:35.902+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:35.946 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:57:35.946 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:35.946 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 23, 2024-09-18T00:57:35.946 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "82a96d76-ba83-49e2-a985-84604bd599ba", 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-18T00:57:07.138732+0000", 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-18T00:57:29.319849+0000", 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-18T00:57:20.258605+0000", 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-18T00:57:09.756380+0000", 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5799936, 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-18T00:57:35.947 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 2, 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "jewel", 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-18T00:57:35.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:21.962521+0000", 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:57:35.949 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:57:35.950 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:57:35.951 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:57:35.951 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:57:35.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.951 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "19", 2024-09-18T00:57:35.951 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:57:35.951 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:57:35.951 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:57:35.952 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:57:35.952 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:57:35.952 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-18T00:57:35.952 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-18T00:57:35.952 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:57:35.952 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:57:35.953 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:57:35.954 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:57:35.955 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 8, 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 8, 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.25, 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-18T00:57:35.956 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:25.721733+0000", 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:57:35.957 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:57:35.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "23", 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 23, 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:57:35.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:57:35.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T00:57:35.961 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 4.9800000190734863, 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 4.9800000190734863, 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.87999999523162842, 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 4.380000114440918, 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 4.380000114440918, 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "3f359ce5-6f6e-4161-8554-8ac028347274", 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:57:35.963 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 0, 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-18T00:57:35.964 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.965 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/199763443", 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/199763443", 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/199763443", 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/199763443", 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "48687611-1556-4b6d-86af-7872494a91d7", 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 17, 2024-09-18T00:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:57:35.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-18T00:57:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.972 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.973 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/345149800", 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/345149800", 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/345149800", 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/345149800", 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "9e6b1f90-ce9b-46d4-92fe-d457f7e415d5", 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 0, 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:57:35.975 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:57:35.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-18T00:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:57:35.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.979 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.979 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:57:35.979 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.979 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.979 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-18T00:57:35.979 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/431239931", 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/431239931", 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/431239931", 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/431239931", 2024-09-18T00:57:35.980 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5", 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:57:35.981 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 20, 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-18T00:57:35.985 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/1775474180", 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/1775474180", 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/1775474180", 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/1775474180", 2024-09-18T00:57:35.986 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "802ccbeb-7351-4b38-832a-695117c7e493", 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:57:35.987 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 20, 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6808", 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6809", 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6810", 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:57:35.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6811", 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6814", 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:57:35.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6815", 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6812", 2024-09-18T00:57:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6813", 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6809/4198401207", 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6811/4198401207", 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6815/4198401207", 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6813/4198401207", 2024-09-18T00:57:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d958b3da-efa1-4d86-b81d-e78874441437", 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:57:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 20, 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6824", 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6825", 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6826", 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6827", 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6830", 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6831", 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6828", 2024-09-18T00:57:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6829", 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6825/2599825823", 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6827/2599825823", 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6831/2599825823", 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6829/2599825823", 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:57:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "51b3cea8-c686-4fd3-b3f4-ba936cf26b64", 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:57:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 0, 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6800", 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6801", 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6802", 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6803", 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6806", 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6807", 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6804", 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6805", 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6801/3550656451", 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6803/3550656451", 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6807/3550656451", 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6805/3550656451", 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:57:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b02ead1c-837a-4b07-9ad1-4e903ed5bc4e", 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 20, 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:57:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6816", 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6817", 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6818", 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6819", 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6822", 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6823", 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:57:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6820", 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6821", 2024-09-18T00:57:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6817/2993352359", 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6819/2993352359", 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6823/2993352359", 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6821/2993352359", 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:57:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.896477+0000", 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:57:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.941756+0000", 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:57:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.560180+0000", 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.077394+0000", 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:57:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.996264+0000", 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:57:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.511686+0000", 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:57:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.411051+0000", 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.888092+0000", 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-18T00:57:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [ 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 2, 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:57:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [ 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 2, 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-18T00:57:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-18T00:57:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-18T00:57:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-18T00:57:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-18T00:57:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-18T00:57:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-18T00:57:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:36.020 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:36.022 INFO:tasks.thrashosds.thrasher:j is {'epoch': 23, 'fsid': '82a96d76-ba83-49e2-a985-84604bd599ba', 'created': '2024-09-18T00:57:07.138732+0000', 'modified': '2024-09-18T00:57:29.319849+0000', 'last_up_change': '2024-09-18T00:57:20.258605+0000', 'last_in_change': '2024-09-18T00:57:09.756380+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T00:57:21.962521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T00:57:25.721733+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3f359ce5-6f6e-4161-8554-8ac028347274', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 199763443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 199763443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 199763443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 199763443}]}, 'public_addr': '172.21.15.42:6801/199763443', 'cluster_addr': '172.21.15.42:6803/199763443', 'heartbeat_back_addr': '172.21.15.42:6807/199763443', 'heartbeat_front_addr': '172.21.15.42:6805/199763443', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '48687611-1556-4b6d-86af-7872494a91d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 345149800}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 345149800}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 345149800}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 345149800}]}, 'public_addr': '172.21.15.42:6809/345149800', 'cluster_addr': '172.21.15.42:6811/345149800', 'heartbeat_back_addr': '172.21.15.42:6815/345149800', 'heartbeat_front_addr': '172.21.15.42:6813/345149800', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9e6b1f90-ce9b-46d4-92fe-d457f7e415d5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 431239931}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 431239931}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 431239931}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 431239931}]}, 'public_addr': '172.21.15.42:6825/431239931', 'cluster_addr': '172.21.15.42:6827/431239931', 'heartbeat_back_addr': '172.21.15.42:6831/431239931', 'heartbeat_front_addr': '172.21.15.42:6829/431239931', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 1775474180}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 1775474180}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 1775474180}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 1775474180}]}, 'public_addr': '172.21.15.42:6817/1775474180', 'cluster_addr': '172.21.15.42:6819/1775474180', 'heartbeat_back_addr': '172.21.15.42:6823/1775474180', 'heartbeat_front_addr': '172.21.15.42:6821/1775474180', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '802ccbeb-7351-4b38-832a-695117c7e493', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6809', 'nonce': 4198401207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6811', 'nonce': 4198401207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6815', 'nonce': 4198401207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6813', 'nonce': 4198401207}]}, 'public_addr': '172.21.15.195:6809/4198401207', 'cluster_addr': '172.21.15.195:6811/4198401207', 'heartbeat_back_addr': '172.21.15.195:6815/4198401207', 'heartbeat_front_addr': '172.21.15.195:6813/4198401207', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd958b3da-efa1-4d86-b81d-e78874441437', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6824', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6825', 'nonce': 2599825823}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6826', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6827', 'nonce': 2599825823}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6830', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6831', 'nonce': 2599825823}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6828', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6829', 'nonce': 2599825823}]}, 'public_addr': '172.21.15.195:6825/2599825823', 'cluster_addr': '172.21.15.195:6827/2599825823', 'heartbeat_back_addr': '172.21.15.195:6831/2599825823', 'heartbeat_front_addr': '172.21.15.195:6829/2599825823', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '51b3cea8-c686-4fd3-b3f4-ba936cf26b64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6800', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6801', 'nonce': 3550656451}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6803', 'nonce': 3550656451}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6807', 'nonce': 3550656451}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6805', 'nonce': 3550656451}]}, 'public_addr': '172.21.15.195:6801/3550656451', 'cluster_addr': '172.21.15.195:6803/3550656451', 'heartbeat_back_addr': '172.21.15.195:6807/3550656451', 'heartbeat_front_addr': '172.21.15.195:6805/3550656451', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b02ead1c-837a-4b07-9ad1-4e903ed5bc4e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6817', 'nonce': 2993352359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6818', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6819', 'nonce': 2993352359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6822', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6823', 'nonce': 2993352359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6820', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6821', 'nonce': 2993352359}]}, 'public_addr': '172.21.15.195:6817/2993352359', 'cluster_addr': '172.21.15.195:6819/2993352359', 'heartbeat_back_addr': '172.21.15.195:6823/2993352359', 'heartbeat_front_addr': '172.21.15.195:6821/2993352359', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.896477+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.941756+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.560180+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.077394+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.996264+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.511686+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.411051+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.888092+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T00:57:36.023 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-18T00:57:36.023 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:36.023 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:36.024 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:57:36.024 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:36.018+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:36.122 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:36.118+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:36.188 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:36.189 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:36.202 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:57:36.223 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:36.218+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:36.312 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:36.313 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:36.324 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:36.318+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:36.325 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:57:36.372 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:57:36.442 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:36.438+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:36.488 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:36.488 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:36.501 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:57:36.526 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:36.522+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:36.627 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:36.622+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:36.673 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:57:36.762 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:36.758+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:36.830 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:36.826+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:36.838 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:36.839 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:36.852 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:57:36.938 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:36.934+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:37.016 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:37.016 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:37.029 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:57:37.074 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:37.070+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:37.133 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:37.126+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:37.203 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:37.203 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:37.218 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:57:37.262 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:37.258+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:37.336 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:37.330+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:37.384 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:37.384 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:37.399 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:57:37.446 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:37.442+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:37.524 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:37.525 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:37.537 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:37.530+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:37.539 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:57:37.639 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:37.634+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:37.671 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:37.671 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:37.685 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:57:37.740 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:37.734+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:37.806 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:37.806 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:37.820 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:57:37.841 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:37.834+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:37.924 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:57:37.945 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:37.945 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:37.959 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:57:37.974 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:37.970+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:38.042 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:38.038+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:38.079 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:38.080 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:38.087 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:57:38.087 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:57:38.093 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:57:38.104 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:57:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:57:38.121 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:57:38.121 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:57:38.139 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:57:38.139 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:57:38.143 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:38.138+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:38.157 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:57:38.157 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:57:38.176 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:57:38.176 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:57:38.197 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:57:38.197 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:57:38.217 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:57:38.217 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:57:38.217 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:57:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:57:38.218 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:57:38.218 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:38.232 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:57:38.278 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:38.274+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:38.345 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:38.338+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:38.351 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:20 1.0 e18/17 1:30306672:devicehealth::main.db.0000000000000000:head v 18'10, mlcod=18'8)", 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.519725+0000", 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.829943369, 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024004669999999999, 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.352 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.519725+0000", 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.519725+0000", 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.519727+0000", 2024-09-18T00:57:38.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9570000000000001e-06 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.519947+0000", 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022030800000000001 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.519977+0000", 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9240000000000001e-05 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.519990+0000", 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3210000000000001e-05 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.520084+0000", 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4246999999999994e-05 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.520114+0000", 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0175000000000002e-05 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.522035+0000", 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001920944 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.522125+0000", 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0385999999999998e-05 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:35 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'17)", 2024-09-18T00:57:38.356 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.551472+0000", 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.798196302999999, 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027913199999999999, 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.551472+0000", 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.551472+0000", 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.551473+0000", 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.313e-06 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.551583+0000", 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010998899999999999 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.551605+0000", 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1987000000000001e-05 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.551617+0000", 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1426e-05 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.551732+0000", 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114879 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.551782+0000", 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0563999999999999e-05 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.554167+0000", 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023849660000000001 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.554263+0000", 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6195999999999999e-05 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.360 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:55 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'33, mlcod=18'29)", 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.594429+0000", 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.755239626, 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00247489, 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.594429+0000", 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.594429+0000", 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.594430+0000", 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5770000000000001e-06 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.594500+0000", 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9356000000000005e-05 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.594528+0000", 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8739999999999999e-05 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.594543+0000", 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4306e-05 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.594622+0000", 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8953000000000001e-05 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.594657+0000", 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5660000000000001e-05 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.596799+0000", 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021418549999999998 2024-09-18T00:57:38.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.596904+0000", 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104443 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:69 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'43, mlcod=18'39)", 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.621738+0000", 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.727930227, 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027185239999999999, 2024-09-18T00:57:38.365 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.621738+0000", 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.621738+0000", 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.621740+0000", 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.556e-06 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.621868+0000", 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000128479 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.621896+0000", 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8036999999999999e-05 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.621915+0000", 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9236999999999999e-05 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.622020+0000", 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104949 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.622059+0000", 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8989999999999998e-05 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.624394+0000", 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023342670000000001 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.624457+0000", 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3009000000000006e-05 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:84 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'54, mlcod=18'50)", 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.651818+0000", 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.697850133999999, 2024-09-18T00:57:38.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026498509999999999, 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.651818+0000", 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.651818+0000", 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.651820+0000", 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6729999999999999e-06 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.651878+0000", 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8483000000000002e-05 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.651904+0000", 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5469e-05 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.651918+0000", 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3896999999999999e-05 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.652037+0000", 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119509 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.652076+0000", 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8393000000000003e-05 2024-09-18T00:57:38.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.654403+0000", 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002327628 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.654468+0000", 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4799000000000001e-05 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:104 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'66, mlcod=18'64)", 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.820065+0000", 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.529603387, 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028424539999999999, 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820065+0000", 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820065+0000", 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820067+0000", 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9240000000000001e-06 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820079+0000", 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.242e-05 2024-09-18T00:57:38.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820110+0000", 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0439000000000001e-05 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820125+0000", 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5217e-05 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820279+0000", 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153855 2024-09-18T00:57:38.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820316+0000", 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7249999999999997e-05 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.822800+0000", 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024835399999999998 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.822907+0000", 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107809 2024-09-18T00:57:38.377 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:105 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'67, mlcod=18'64)", 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.820444+0000", 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.529224046, 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002561486, 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820444+0000", 2024-09-18T00:57:38.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820444+0000", 2024-09-18T00:57:38.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820446+0000", 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820458+0000", 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.254e-05 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820499+0000", 2024-09-18T00:57:38.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0843999999999997e-05 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820516+0000", 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6702999999999999e-05 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.821052+0000", 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053601600000000005 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.821102+0000", 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9820000000000001e-05 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.822919+0000", 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00181767 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.823006+0000", 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6457000000000002e-05 2024-09-18T00:57:38.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.383 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.383 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.383 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:106 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'68, mlcod=18'64)", 2024-09-18T00:57:38.383 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.820927+0000", 2024-09-18T00:57:38.383 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.528741036, 2024-09-18T00:57:38.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003515673, 2024-09-18T00:57:38.383 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820927+0000", 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820927+0000", 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.820929+0000", 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.22e-06 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.821000+0000", 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1132999999999998e-05 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.821023+0000", 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3195999999999999e-05 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.821035+0000", 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1807000000000001e-05 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.821785+0000", 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00074985799999999995 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.821819+0000", 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4363000000000003e-05 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.824381+0000", 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025620090000000001 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.824443+0000", 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2087000000000004e-05 2024-09-18T00:57:38.387 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:107 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'69, mlcod=18'64)", 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.822255+0000", 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.527412898, 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033471070000000002, 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.822255+0000", 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.822255+0000", 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.822258+0000", 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0700000000000001e-06 2024-09-18T00:57:38.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.822273+0000", 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5252999999999999e-05 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.822335+0000", 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2201999999999998e-05 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.822358+0000", 2024-09-18T00:57:38.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2563999999999999e-05 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.823075+0000", 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00071765800000000003 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.823115+0000", 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9520000000000001e-05 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.825526+0000", 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024115510000000001 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.825603+0000", 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6289000000000001e-05 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:109 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'71, mlcod=18'64)", 2024-09-18T00:57:38.392 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.824048+0000", 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.525620104, 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030829020000000002, 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.824048+0000", 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.824048+0000", 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.824050+0000", 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.764e-06 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.824121+0000", 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0709000000000004e-05 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.824146+0000", 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5293e-05 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.824163+0000", 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.719e-05 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.824521+0000", 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035740799999999998 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.824569+0000", 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8001999999999999e-05 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.827028+0000", 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024595670000000002 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.827131+0000", 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102969 2024-09-18T00:57:38.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:111 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'73, mlcod=18'67)", 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.825782+0000", 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.523886731999999, 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023389600000000002, 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.825782+0000", 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.825782+0000", 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.825783+0000", 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.825791+0000", 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6330000000000002e-06 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.825807+0000", 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5716999999999999e-05 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.825825+0000", 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7693000000000002e-05 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.826395+0000", 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00057078300000000005 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.826428+0000", 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2598999999999997e-05 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.828050+0000", 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016218949999999999 2024-09-18T00:57:38.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.828121+0000", 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0640000000000001e-05 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:112 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'74, mlcod=18'67)", 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.826202+0000", 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.523466105000001, 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003192088, 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.401 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.826202+0000", 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.826202+0000", 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.826203+0000", 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0729999999999999e-06 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.826260+0000", 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6379999999999999e-05 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.826275+0000", 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5659999999999999e-05 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.826285+0000", 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8430000000000005e-06 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.827302+0000", 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010166859999999999 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.827344+0000", 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1640000000000001e-05 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.829310+0000", 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019663800000000002 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.829394+0000", 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4425999999999994e-05 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.405 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:114 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'76, mlcod=18'69)", 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:23.828390+0000", 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.521278127, 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024511060000000002, 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.828390+0000", 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.828390+0000", 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.828391+0000", 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.828404+0000", 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.288e-05 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.828425+0000", 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1080000000000001e-05 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.828444+0000", 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9069999999999999e-05 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.828737+0000", 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029282100000000002 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.828779+0000", 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1359000000000002e-05 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.830778+0000", 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019993400000000001 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:23.830841+0000", 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3425999999999999e-05 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.409 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:153 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'107, mlcod=19'105)", 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:25.089995+0000", 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.259672994000001, 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0049939579999999997, 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.089995+0000", 2024-09-18T00:57:38.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.089995+0000", 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.089997+0000", 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0660000000000002e-06 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.090038+0000", 2024-09-18T00:57:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0735999999999998e-05 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.090069+0000", 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0700000000000001e-05 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.090087+0000", 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7672999999999998e-05 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.090224+0000", 2024-09-18T00:57:38.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013747600000000001 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.090269+0000", 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4660000000000003e-05 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.094885+0000", 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046160109999999997 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.094989+0000", 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010463600000000001 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:154 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'108, mlcod=19'105)", 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:25.090499+0000", 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.259169328, 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045880349999999999, 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.414 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.090499+0000", 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.090499+0000", 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.090501+0000", 2024-09-18T00:57:38.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8560000000000001e-06 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.090514+0000", 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3013999999999999e-05 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.090538+0000", 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.402e-05 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.090582+0000", 2024-09-18T00:57:38.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3699999999999998e-05 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.091068+0000", 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048643900000000002 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.091105+0000", 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6943000000000002e-05 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.094997+0000", 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038916699999999999 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.095087+0000", 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0393e-05 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:155 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'109, mlcod=19'105)", 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:25.091244+0000", 2024-09-18T00:57:38.418 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.258424293000001, 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006156824, 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.091244+0000", 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.091244+0000", 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.091249+0000", 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4630000000000003e-06 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.091453+0000", 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000204506 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.091506+0000", 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2772000000000002e-05 2024-09-18T00:57:38.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.091527+0000", 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1503000000000001e-05 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.091829+0000", 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030203499999999998 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.091866+0000", 2024-09-18T00:57:38.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7060000000000001e-05 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.097293+0000", 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054263159999999996 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.097401+0000", 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000108169 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:156 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'110, mlcod=19'105)", 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:25.092120+0000", 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.257548208999999, 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005366867, 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092120+0000", 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092120+0000", 2024-09-18T00:57:38.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092122+0000", 2024-09-18T00:57:38.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.626e-06 2024-09-18T00:57:38.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092146+0000", 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4620000000000001e-05 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092181+0000", 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4297e-05 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092199+0000", 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8179e-05 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.093085+0000", 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00088657099999999995 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.093128+0000", 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2592999999999997e-05 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.097405+0000", 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0042770739999999996 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.097487+0000", 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1907e-05 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:157 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'111, mlcod=19'105)", 2024-09-18T00:57:38.427 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:25.092736+0000", 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.256932793000001, 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0048911459999999999, 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092736+0000", 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092736+0000", 2024-09-18T00:57:38.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092737+0000", 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.463e-06 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092755+0000", 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8266000000000001e-05 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092782+0000", 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.686e-05 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.092798+0000", 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6016e-05 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.093965+0000", 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0011667719999999999 2024-09-18T00:57:38.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.094026+0000", 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1309999999999994e-05 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.097494+0000", 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034675840000000001 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.097627+0000", 2024-09-18T00:57:38.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013287499999999999 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:158 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'112, mlcod=19'105)", 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:25.093478+0000", 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.256190504999999, 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0042421079999999996, 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.432 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.093478+0000", 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.093478+0000", 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.093480+0000", 2024-09-18T00:57:38.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3159999999999999e-06 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.093657+0000", 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000176842 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.093688+0000", 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0877000000000003e-05 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.093706+0000", 2024-09-18T00:57:38.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8459999999999999e-05 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.095196+0000", 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014894330000000001 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.095241+0000", 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5346999999999998e-05 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.097630+0000", 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023888270000000001 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.097720+0000", 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0006000000000005e-05 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4204.0:159 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'113, mlcod=19'105)", 2024-09-18T00:57:38.436 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:25.094517+0000", 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.255150926000001, 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0039461599999999998, 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.094517+0000", 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.094517+0000", 2024-09-18T00:57:38.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.094519+0000", 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.094535+0000", 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5837000000000001e-05 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.094557+0000", 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2849000000000001e-05 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.094569+0000", 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1250000000000001e-05 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.096377+0000", 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001808715 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.096442+0000", 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4319999999999994e-05 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.098388+0000", 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019466660000000001 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:25.098464+0000", 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5283000000000004e-05 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.440 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.441 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.441 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.441 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:38.441 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:57:38.442 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:57:38.486 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:38.482+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:38.547 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:38.542+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:38.612 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:57:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:57:38.612 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:38.626 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:57:38.648 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:38.642+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:38.746 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:38.747 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:57:38.748 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:57:38.748 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:57:38.748 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:38.749 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:38.742+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:38.760 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:57:38.850 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:38.846+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:38.896 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:57:38.931 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4226.0:5 2.2 e22/20 2:5cea7035:::rbd_info:head v 22'1, mlcod=0'0)", 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:28.327364+0000", 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.603621308999999, 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023517669999999998, 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327364+0000", 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327364+0000", 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327364+0000", 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2e-07 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327376+0000", 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2150000000000001e-05 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327403+0000", 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6322999999999999e-05 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327420+0000", 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7033000000000001e-05 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327556+0000", 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000135816 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327632+0000", 2024-09-18T00:57:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6499000000000001e-05 2024-09-18T00:57:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.329585+0000", 2024-09-18T00:57:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019527730000000001 2024-09-18T00:57:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.329716+0000", 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000130653 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4226.0:7 2.2 e23/20 2:5cea7035:::rbd_info:head v 23'2, mlcod=0'0)", 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:29.331452+0000", 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.5995326429999999, 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021910789999999999, 2024-09-18T00:57:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331452+0000", 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331452+0000", 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331453+0000", 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8400000000000005e-07 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331463+0000", 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0380000000000001e-05 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331522+0000", 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8792999999999998e-05 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331545+0000", 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3062999999999999e-05 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331704+0000", 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015847499999999999 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331760+0000", 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6672999999999998e-05 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.333514+0000", 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001753962 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.333644+0000", 2024-09-18T00:57:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012924900000000001 2024-09-18T00:57:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:38.942 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:38.946 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:57:38.990 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:38.986+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:39.054 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:39.050+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:39.111 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:39.112 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:57:39.112 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:57:39.112 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:39.125 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:57:39.170 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:39.166+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:39.255 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:39.250+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:39.290 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:39.291 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:57:39.291 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:57:39.291 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:57:39.291 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:39.305 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:57:39.390 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:39.386+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:39.457 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:39.450+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:39.469 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:39.469 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:57:39.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:57:39.470 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [] 2024-09-18T00:57:39.470 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:39.483 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:57:39.557 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:39.550+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:39.606 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:39.607 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:39.607 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:39.607 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:39.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:57:39.662 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:39.658+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:39.760 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:39.754+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:39.788 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:57:39.874 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:39.870+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:39.961 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:39.961 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:10 3.1c e27/24 3:3831950c:::rbd_directory:head v 27'1, mlcod=0'0)", 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:38.906907+0000", 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.053922477, 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094127999999999996, 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.906907+0000", 2024-09-18T00:57:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.906907+0000", 2024-09-18T00:57:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.906907+0000", 2024-09-18T00:57:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5099999999999999e-07 2024-09-18T00:57:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.906915+0000", 2024-09-18T00:57:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9459999999999998e-06 2024-09-18T00:57:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.906925+0000", 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6549999999999999e-06 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.906934+0000", 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6740000000000003e-06 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.907000+0000", 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6667000000000004e-05 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.907031+0000", 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1000999999999997e-05 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.907801+0000", 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00076965099999999999 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.907848+0000", 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7034999999999999e-05 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:13 3.1b e27/24 3:d91a31cf:::rbd_object_map.117c6d19cb58:head v 27'1, mlcod=0'0)", 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:38.920768+0000", 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.0400605569999999, 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001166647, 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.920768+0000", 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.920768+0000", 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.920769+0000", 2024-09-18T00:57:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4700000000000002e-07 2024-09-18T00:57:39.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:39.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.920777+0000", 2024-09-18T00:57:39.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8809999999999996e-06 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.920802+0000", 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5508999999999999e-05 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.920819+0000", 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7025e-05 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.920909+0000", 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9877999999999996e-05 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.920931+0000", 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2206999999999999e-05 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.921880+0000", 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094871799999999998 2024-09-18T00:57:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:38.921935+0000", 2024-09-18T00:57:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5081999999999999e-05 2024-09-18T00:57:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:39.973 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:39.973 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:39.958+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:39.977 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:57:40.063 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:40.058+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:40.137 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:40.138 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:40.150 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:57:40.163 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:40.158+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:40.237 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:57:40.264 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:40.258+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:40.282 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:57:40.364 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:40.358+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:40.379 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:57:40.379 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:57:40.395 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:57:40.396 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:57:40.413 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:57:40.413 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:57:40.430 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:57:40.430 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:57:40.446 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:40.446 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:40.451 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:57:40.451 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:57:40.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:57:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:57:40.474 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:57:40.474 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:40.470+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:40.494 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:57:40.495 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:57:40.515 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:57:40.516 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:57:40.571 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:40.571 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:40.571 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:40.566+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:40.584 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:57:40.668 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:40.666+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:40.741 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:40.741 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:40.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:57:40.802 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:40.798+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:40.843 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:40.843 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:40.869 INFO:teuthology.orchestra.run.smithi042.stdout:3 writesame 0xe9be08b thru 0xe9be3d0 data_size 0x1(0x346 bytes) 2024-09-18T00:57:40.869 INFO:teuthology.orchestra.run.smithi042.stdout:6 write 0x600ff2c thru 0x601cf21 (0xcff6 bytes) 2024-09-18T00:57:40.869 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:40.862+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:40.878 INFO:teuthology.orchestra.run.smithi042.stdout:7 read 0x97bed8e thru 0x97cac46 (0xbeb9 bytes) 2024-09-18T00:57:40.878 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xcdd9ede 2024-09-18T00:57:40.880 INFO:teuthology.orchestra.run.smithi042.stdout:11 trunc from 0xe9be3d1 to 0xcdd9ede 2024-09-18T00:57:40.882 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:40.882 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:40.894 INFO:teuthology.orchestra.run.smithi042.stdout:12 read 0x183d970 thru 0x1843de6 (0x6477 bytes) 2024-09-18T00:57:40.894 INFO:teuthology.orchestra.run.smithi042.stdout:15 writesame 0xc00f263 thru 0xc0149ce data_size 0x1e(0x576c bytes) 2024-09-18T00:57:40.897 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:57:40.901 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:40.901 INFO:teuthology.orchestra.run.smithi042.stdout:18 writesame 0x6392962 thru 0x6395b3f data_size 0xd(0x31de bytes) 2024-09-18T00:57:40.905 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:40.905 INFO:teuthology.orchestra.run.smithi042.stdout:20 punch from 0xaf9c037 to 0xafaa5e4, (0xe5ad bytes) 2024-09-18T00:57:40.905 INFO:teuthology.orchestra.run.smithi042.stdout:21 write 0x9273cf2 thru 0x927f50d (0xb81c bytes) 2024-09-18T00:57:40.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:57:40.970 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:40.966+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:41.023 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:41.023 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:41.024 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:57:41.024 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:57:41.024 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-18T00:57:41.024 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T00:57:41.024 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T00:57:41.024 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-18T00:57:41.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:57:41.072 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:41.066+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:41.116 INFO:teuthology.orchestra.run.smithi042.stdout:22 write 0x37b4f3c thru 0x37bf349 (0xa40e bytes) 2024-09-18T00:57:41.125 INFO:teuthology.orchestra.run.smithi042.stdout:23 trunc from 0xcdd9ede to 0x9886807 2024-09-18T00:57:41.143 INFO:teuthology.orchestra.run.smithi042.stdout:24 writesame 0x62bf0fa thru 0x62c38b2 data_size 0x2b(0x47b9 bytes) 2024-09-18T00:57:41.146 INFO:teuthology.orchestra.run.smithi042.stdout:25 writesame 0x783ce64 thru 0x784b298 data_size 0x2f(0xe435 bytes) 2024-09-18T00:57:41.146 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:41.147 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T00:57:41.147 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:41.155 INFO:teuthology.orchestra.run.smithi042.stdout:27 writesame 0x3b87736 thru 0x3b91f4a data_size 0x15(0xa815 bytes) 2024-09-18T00:57:41.163 INFO:teuthology.orchestra.run.smithi042.stdout:28 read 0x962eb99 thru 0x963e950 (0xfdb8 bytes) 2024-09-18T00:57:41.163 INFO:teuthology.orchestra.run.smithi042.stdout:29 read 0x4a34797 thru 0x4a3bb8e (0x73f8 bytes) 2024-09-18T00:57:41.164 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xe935e05 2024-09-18T00:57:41.170 INFO:teuthology.orchestra.run.smithi042.stdout:30 trunc from 0x9886807 to 0xe935e05 2024-09-18T00:57:41.172 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:41.166+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:41.177 INFO:teuthology.orchestra.run.smithi042.stdout:31 read 0x3062733 thru 0x3066fac (0x487a bytes) 2024-09-18T00:57:41.177 INFO:teuthology.orchestra.run.smithi042.stdout:32 writesame 0xa84b0d8 thru 0xa85356b data_size 0xa(0x8494 bytes) 2024-09-18T00:57:41.189 INFO:teuthology.orchestra.run.smithi042.stdout:33 trunc from 0xe935e05 to 0xd996be2 2024-09-18T00:57:41.197 INFO:teuthology.orchestra.run.smithi042.stdout:35 writesame 0xcb09bec thru 0xcb0b6d6 data_size 0x1(0x1aeb bytes) 2024-09-18T00:57:41.199 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:41.199 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:41.204 INFO:teuthology.orchestra.run.smithi042.stdout:36 trunc from 0xd996be2 to 0xdd41405 2024-09-18T00:57:41.211 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:41.211 INFO:teuthology.orchestra.run.smithi042.stdout:39 read 0x1e52bc0 thru 0x1e61bbd (0xeffe bytes) 2024-09-18T00:57:41.212 INFO:teuthology.orchestra.run.smithi042.stdout:40 punch from 0xc22ea2a to 0xc2321d8, (0x37ae bytes) 2024-09-18T00:57:41.212 INFO:teuthology.orchestra.run.smithi042.stdout:41 punch from 0xa35af67 to 0xa368cc0, (0xdd59 bytes) 2024-09-18T00:57:41.212 INFO:teuthology.orchestra.run.smithi042.stdout:42 write 0x4af5946 thru 0x4afcc54 (0x730f bytes) 2024-09-18T00:57:41.213 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:57:41.223 INFO:teuthology.orchestra.run.smithi042.stdout:44 trunc from 0xdd41405 to 0xa3f5cc3 2024-09-18T00:57:41.241 INFO:teuthology.orchestra.run.smithi042.stdout:45 read 0x3ee59a4 thru 0x3ee7c4a (0x22a7 bytes) 2024-09-18T00:57:41.242 INFO:teuthology.orchestra.run.smithi042.stdout:46 write 0x4b4e37e thru 0x4b5dd2d (0xf9b0 bytes) 2024-09-18T00:57:41.246 INFO:teuthology.orchestra.run.smithi042.stdout:47 trunc from 0xa3f5cc3 to 0xb0d9fe1 2024-09-18T00:57:41.255 INFO:teuthology.orchestra.run.smithi042.stdout:48 punch from 0x24e54ff to 0x24f1a81, (0xc582 bytes) 2024-09-18T00:57:41.255 INFO:teuthology.orchestra.run.smithi042.stdout:52 trunc from 0xb0d9fe1 to 0xb39d431 2024-09-18T00:57:41.264 INFO:teuthology.orchestra.run.smithi042.stdout:54 read 0x1c121c2 thru 0x1c1d30e (0xb14d bytes) 2024-09-18T00:57:41.264 INFO:teuthology.orchestra.run.smithi042.stdout:55 write 0x32aa60 thru 0x332839 (0x7dda bytes) 2024-09-18T00:57:41.273 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:41.266+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:41.280 INFO:teuthology.orchestra.run.smithi042.stdout:57 writesame 0xc878826 thru 0xc87daa0 data_size 0x29(0x527b bytes) 2024-09-18T00:57:41.280 INFO:teuthology.orchestra.run.smithi042.stdout:58 punch from 0xc69e4fb to 0xc6a693d, (0x8442 bytes) 2024-09-18T00:57:41.281 INFO:teuthology.orchestra.run.smithi042.stdout:59 read 0x668178f thru 0x668d89c (0xc10e bytes) 2024-09-18T00:57:41.281 INFO:teuthology.orchestra.run.smithi042.stdout:60 read 0x5041f2b thru 0x5051c6b (0xfd41 bytes) 2024-09-18T00:57:41.281 INFO:teuthology.orchestra.run.smithi042.stdout:63 punch from 0xa33e5f4 to 0xa33f6ae, (0x10ba bytes) 2024-09-18T00:57:41.292 INFO:teuthology.orchestra.run.smithi042.stdout:64 trunc from 0xc87daa1 to 0x1d11e03 2024-09-18T00:57:41.341 INFO:teuthology.orchestra.run.smithi042.stdout:65 write 0xe294626 thru 0xe2a0411 (0xbdec bytes) 2024-09-18T00:57:41.349 INFO:teuthology.orchestra.run.smithi042.stdout:66 punch from 0x82e7fa6 to 0x82f68db, (0xe935 bytes) 2024-09-18T00:57:41.349 INFO:teuthology.orchestra.run.smithi042.stdout:67 writesame 0xb9d5288 thru 0xb9d9b60 data_size 0x1(0x48d9 bytes) 2024-09-18T00:57:41.358 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:41.358 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:41.368 INFO:teuthology.orchestra.run.smithi042.stdout:68 read 0xaf3d1b8 thru 0xaf4cc9a (0xfae3 bytes) 2024-09-18T00:57:41.368 INFO:teuthology.orchestra.run.smithi042.stdout:69 punch from 0x2eda1ba to 0x2ede6bd, (0x4503 bytes) 2024-09-18T00:57:41.368 INFO:teuthology.orchestra.run.smithi042.stdout:70 write 0x698788d thru 0x699069d (0x8e11 bytes) 2024-09-18T00:57:41.372 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:57:41.374 INFO:teuthology.orchestra.run.smithi042.stdout:71 read 0x7c9fcf6 thru 0x7ca778f (0x7a9a bytes) 2024-09-18T00:57:41.375 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:41.370+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:41.375 INFO:teuthology.orchestra.run.smithi042.stdout:72 writesame 0x1a744df thru 0x1a80878 data_size 0x1(0xc39a bytes) 2024-09-18T00:57:41.414 INFO:teuthology.orchestra.run.smithi042.stdout:73 read 0xb308eb4 thru 0xb318cec (0xfe39 bytes) 2024-09-18T00:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout:75 writesame 0xe9f8d8a thru 0xea00872 data_size 0x23(0x7ae9 bytes) 2024-09-18T00:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout:77 write 0xe34373b thru 0xe346c65 (0x352b bytes) 2024-09-18T00:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout:78 trunc from 0xea00873 to 0xa35377d 2024-09-18T00:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout:79 punch from 0x2f4c3de to 0x2f5a881, (0xe4a3 bytes) 2024-09-18T00:57:41.467 INFO:teuthology.orchestra.run.smithi042.stdout:81 write 0xc302928 thru 0xc30d01a (0xa6f3 bytes) 2024-09-18T00:57:41.474 INFO:teuthology.orchestra.run.smithi042.stdout:82 read 0x63458c5 thru 0x634a0cf (0x480b bytes) 2024-09-18T00:57:41.474 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:41.470+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:41.489 INFO:teuthology.orchestra.run.smithi042.stdout:83 write 0xe5292f6 thru 0xe536636 (0xd341 bytes) 2024-09-18T00:57:41.496 INFO:teuthology.orchestra.run.smithi042.stdout:84 punch from 0xa66fc4c to 0xa67d8c6, (0xdc7a bytes) 2024-09-18T00:57:41.496 INFO:teuthology.orchestra.run.smithi042.stdout:85 punch from 0xbcfd1d7 to 0xbcfd530, (0x359 bytes) 2024-09-18T00:57:41.496 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:41.496 INFO:teuthology.orchestra.run.smithi042.stdout:88 punch from 0x156a12f to 0x156a64c, (0x51d bytes) 2024-09-18T00:57:41.496 INFO:teuthology.orchestra.run.smithi042.stdout:89 read 0x315edc6 thru 0x316dbf3 (0xee2e bytes) 2024-09-18T00:57:41.496 INFO:teuthology.orchestra.run.smithi042.stdout:90 punch from 0xac760e to 0xaccdd7, (0x57c9 bytes) 2024-09-18T00:57:41.496 INFO:teuthology.orchestra.run.smithi042.stdout:91 write 0x2b3da75 thru 0x2b41e07 (0x4393 bytes) 2024-09-18T00:57:41.503 INFO:teuthology.orchestra.run.smithi042.stdout:92 read 0xe3ae817 thru 0xe3bbd47 (0xd531 bytes) 2024-09-18T00:57:41.503 INFO:teuthology.orchestra.run.smithi042.stdout:93 writesame 0xcb9e44c thru 0xcbae408 data_size 0x8b(0xffbd bytes) 2024-09-18T00:57:41.508 INFO:teuthology.orchestra.run.smithi042.stdout:94 writesame 0xc930399 thru 0xc9332fe data_size 0x1(0x2f66 bytes) 2024-09-18T00:57:41.512 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:41.512 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:41.523 INFO:teuthology.orchestra.run.smithi042.stdout:96 write 0x53814b4 thru 0x538c3ca (0xaf17 bytes) 2024-09-18T00:57:41.526 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:57:41.529 INFO:teuthology.orchestra.run.smithi042.stdout:97 punch from 0x930265 to 0x9358fd, (0x5698 bytes) 2024-09-18T00:57:41.530 INFO:teuthology.orchestra.run.smithi042.stdout:98 write 0xb57997c thru 0xb587ba4 (0xe229 bytes) 2024-09-18T00:57:41.564 INFO:teuthology.orchestra.run.smithi042.stdout:99 trunc from 0xe536637 to 0x2a8ef3c 2024-09-18T00:57:41.575 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:41.570+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:41.587 INFO:teuthology.orchestra.run.smithi042.stdout:100 write 0x1c68fb5 thru 0x1c74108 (0xb154 bytes) 2024-09-18T00:57:41.611 INFO:teuthology.orchestra.run.smithi042.stdout:101 trunc from 0x2a8ef3c to 0x9ae738a 2024-09-18T00:57:41.649 INFO:teuthology.orchestra.run.smithi042.stdout:105 trunc from 0x9ae738a to 0x2c936c 2024-09-18T00:57:41.653 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:41.653 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:41.670 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:57:41.710 INFO:teuthology.orchestra.run.smithi042.stdout:106 write 0x90c34bf thru 0x90cccba (0x97fc bytes) 2024-09-18T00:57:41.714 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:41.710+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:41.722 INFO:teuthology.orchestra.run.smithi042.stdout:107 trunc from 0x90cccbb to 0xb2ccb9d 2024-09-18T00:57:41.733 INFO:teuthology.orchestra.run.smithi042.stdout:108 read 0x26d4536 thru 0x26e0310 (0xbddb bytes) 2024-09-18T00:57:41.734 INFO:teuthology.orchestra.run.smithi042.stdout:109 read 0x3c79995 thru 0x3c806e6 (0x6d52 bytes) 2024-09-18T00:57:41.761 INFO:teuthology.orchestra.run.smithi042.stdout:111 trunc from 0xb2ccb9d to 0x9086aa 2024-09-18T00:57:41.776 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:41.770+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:41.788 INFO:teuthology.orchestra.run.smithi042.stdout:112 trunc from 0x9086aa to 0xaee46d 2024-09-18T00:57:41.790 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:41.790 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:41.805 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:57:41.820 INFO:teuthology.orchestra.run.smithi042.stdout:114 trunc from 0xaee46d to 0x7c5d689 2024-09-18T00:57:41.833 INFO:teuthology.orchestra.run.smithi042.stdout:115 write 0x394ce5c thru 0x394fd3c (0x2ee1 bytes) 2024-09-18T00:57:41.840 INFO:teuthology.orchestra.run.smithi042.stdout:117 punch from 0x2c79fc1 to 0x2c7b599, (0x15d8 bytes) 2024-09-18T00:57:41.860 INFO:teuthology.orchestra.run.smithi042.stdout:118 write 0xa1b9672 thru 0xa1c6619 (0xcfa8 bytes) 2024-09-18T00:57:41.868 INFO:teuthology.orchestra.run.smithi042.stdout:122 write 0x77e0759 thru 0x77ee56b (0xde13 bytes) 2024-09-18T00:57:41.875 INFO:teuthology.orchestra.run.smithi042.stdout:123 read 0x6af7067 thru 0x6b03821 (0xc7bb bytes) 2024-09-18T00:57:41.875 INFO:teuthology.orchestra.run.smithi042.stdout:124 read 0x7530503 thru 0x75329e0 (0x24de bytes) 2024-09-18T00:57:41.876 INFO:teuthology.orchestra.run.smithi042.stdout:125 punch from 0x804b932 to 0x805a41f, (0xeaed bytes) 2024-09-18T00:57:41.876 INFO:teuthology.orchestra.run.smithi042.stdout:126 write 0x3a7709e thru 0x3a7a2a4 (0x3207 bytes) 2024-09-18T00:57:41.877 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:41.870+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:41.879 INFO:teuthology.orchestra.run.smithi042.stdout:129 write 0x209fc62 thru 0x20a5ad1 (0x5e70 bytes) 2024-09-18T00:57:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:41.885 INFO:teuthology.orchestra.run.smithi042.stdout:131 writesame 0x7e6028a thru 0x7e6d992 data_size 0x1(0xd709 bytes) 2024-09-18T00:57:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:57:41.932 INFO:teuthology.orchestra.run.smithi042.stdout:133 trunc from 0xa1c661a to 0xbd5a233 2024-09-18T00:57:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:134 write 0x12b4277 thru 0x12bd07f (0x8e09 bytes) 2024-09-18T00:57:41.952 INFO:teuthology.orchestra.run.smithi042.stdout:135 read 0xa86bba9 thru 0xa874e7a (0x92d2 bytes) 2024-09-18T00:57:41.955 INFO:teuthology.orchestra.run.smithi042.stdout:137 trunc from 0xbd5a233 to 0xa235455 2024-09-18T00:57:41.967 INFO:teuthology.orchestra.run.smithi042.stdout:138 read 0x44da942 thru 0x44db3f4 (0xab3 bytes) 2024-09-18T00:57:41.967 INFO:teuthology.orchestra.run.smithi042.stdout:139 read 0x9d7eb76 thru 0x9d8358f (0x4a1a bytes) 2024-09-18T00:57:41.967 INFO:teuthology.orchestra.run.smithi042.stdout:140 write 0x478354b thru 0x4788e37 (0x58ed bytes) 2024-09-18T00:57:41.972 INFO:teuthology.orchestra.run.smithi042.stdout:141 write 0x90d3004 thru 0x90debfe (0xbbfb bytes) 2024-09-18T00:57:41.974 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:41.974 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:41.977 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:41.970+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:41.978 INFO:teuthology.orchestra.run.smithi042.stdout:142 write 0xee158e thru 0xee2bea (0x165d bytes) 2024-09-18T00:57:41.984 INFO:teuthology.orchestra.run.smithi042.stdout:143 read 0x9cd185a thru 0x9cd6360 (0x4b07 bytes) 2024-09-18T00:57:41.984 INFO:teuthology.orchestra.run.smithi042.stdout:144 write 0xa1d542d thru 0xa1dc019 (0x6bed bytes) 2024-09-18T00:57:41.987 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:41.987 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:41.989 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:57:42.004 INFO:teuthology.orchestra.run.smithi042.stdout:151 trunc from 0xa235455 to 0x2011c80 2024-09-18T00:57:42.058 INFO:teuthology.orchestra.run.smithi042.stdout:152 write 0xa91798b thru 0xa924418 (0xca8e bytes) 2024-09-18T00:57:42.065 INFO:teuthology.orchestra.run.smithi042.stdout:153 clone 1 order 19 su 8192 sc 5 2024-09-18T00:57:42.078 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:42.074+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:42.112 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:42.113 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:42.128 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:57:42.214 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:42.210+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:42.279 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:42.274+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:42.288 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:42.288 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:42.302 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:57:42.390 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:42.386+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:42.471 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:42.471 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:42.481 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:42.474+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:42.485 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:57:42.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:57:42.582 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:42.578+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:42.654 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:42.654 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:42.668 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:57:42.694 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:57:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:57:42.710 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:42.706+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:42.711 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:57:42.711 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:57:42.729 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:57:42.729 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:57:42.746 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:57:42.746 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:57:42.767 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:57:42.767 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:57:42.783 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:42.778+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:42.787 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:57:42.787 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:57:42.809 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:57:42.809 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:57:42.830 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:57:42.830 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4226.0:5 2.2 e22/20 2:5cea7035:::rbd_info:head v 22'1, mlcod=0'0)", 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:28.327364+0000", 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.504920740999999, 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023517669999999998, 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:42.833 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327364+0000", 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327364+0000", 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327364+0000", 2024-09-18T00:57:42.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2e-07 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327376+0000", 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2150000000000001e-05 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327403+0000", 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6322999999999999e-05 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327420+0000", 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7033000000000001e-05 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327556+0000", 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000135816 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.327632+0000", 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6499000000000001e-05 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.329585+0000", 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019527730000000001 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:28.329716+0000", 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000130653 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.837 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4226.0:7 2.2 e23/20 2:5cea7035:::rbd_info:head v 23'2, mlcod=0'0)", 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:29.331452+0000", 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.500832075, 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021910789999999999, 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331452+0000", 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:42.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331452+0000", 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331453+0000", 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8400000000000005e-07 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331463+0000", 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0380000000000001e-05 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331522+0000", 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8792999999999998e-05 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331545+0000", 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3062999999999999e-05 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331704+0000", 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015847499999999999 2024-09-18T00:57:42.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.331760+0000", 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6672999999999998e-05 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.333514+0000", 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001753962 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:29.333644+0000", 2024-09-18T00:57:42.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012924900000000001 2024-09-18T00:57:42.842 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:42.842 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:42.842 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:42.842 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:42.842 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:42.842 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:42.847 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:57:42.894 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:42.890+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:42.974 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:42.974 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:57:42.974 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:57:42.974 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:42.985 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:42.978+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:42.988 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:57:43.087 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:43.082+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:43.155 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:43.155 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:57:43.155 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:57:43.155 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:57:43.155 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:43.169 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:57:43.212 INFO:teuthology.orchestra.run.smithi042.stdout:154 write 0x4485b68 thru 0x4491fa7 (0xc440 bytes) 2024-09-18T00:57:43.214 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:43.210+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:43.236 INFO:teuthology.orchestra.run.smithi042.stdout:156 write 0x6236d26 thru 0x623fdb9 (0x9094 bytes) 2024-09-18T00:57:43.254 INFO:teuthology.orchestra.run.smithi042.stdout:158 writesame 0xa12501d thru 0xa12f8c3 data_size 0xb(0xa8a7 bytes) 2024-09-18T00:57:43.271 INFO:teuthology.orchestra.run.smithi042.stdout:161 write 0x6f9bcf2 thru 0x6fa6115 (0xa424 bytes) 2024-09-18T00:57:43.288 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:43.282+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:43.289 INFO:teuthology.orchestra.run.smithi042.stdout:162 write 0x702f096 thru 0x703af1b (0xbe86 bytes) 2024-09-18T00:57:43.298 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:43.298 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:57:43.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:57:43.298 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [] 2024-09-18T00:57:43.298 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:163 write 0xc406ab9 thru 0xc411726 (0xac6e bytes) 2024-09-18T00:57:43.313 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:57:43.322 INFO:teuthology.orchestra.run.smithi042.stdout:166 read 0xb7dd887 thru 0xb7e7482 (0x9bfc bytes) 2024-09-18T00:57:43.326 INFO:teuthology.orchestra.run.smithi042.stdout:167 trunc from 0xc411727 to 0xafc993a 2024-09-18T00:57:43.342 INFO:teuthology.orchestra.run.smithi042.stdout:168 writesame 0x195e845 thru 0x195f789 data_size 0x1(0xf45 bytes) 2024-09-18T00:57:43.348 INFO:teuthology.orchestra.run.smithi042.stdout:169 write 0x5e8269a thru 0x5e89850 (0x71b7 bytes) 2024-09-18T00:57:43.357 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:43.367 INFO:teuthology.orchestra.run.smithi042.stdout:172 trunc from 0xafc993a to 0x5e71d08 2024-09-18T00:57:43.389 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:43.386+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:173 punch from 0x2dbabde to 0x2dbf021, (0x4443 bytes) 2024-09-18T00:57:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:175 read 0x326f958 thru 0x327cef7 (0xd5a0 bytes) 2024-09-18T00:57:43.391 INFO:teuthology.orchestra.run.smithi042.stdout:176 clone 2 order 21 su 32768 sc 7 2024-09-18T00:57:43.455 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:43.455 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:43.455 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:43.455 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:43.468 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:57:43.490 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:43.486+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:43.591 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:43.586+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:43.635 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:43.635 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:43.635 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:43.635 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:43.635 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:43.649 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:57:43.694 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:43.690+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:43.793 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:43.786+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:43.826 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:57:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:57:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:447 3.1b e29/24 3:d91a31cf:::rbd_object_map.117c6d19cb58:head v 29'82, mlcod=28'80)", 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:42.938855+0000", 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.88289668700000001, 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026937979999999999, 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:42.938855+0000", 2024-09-18T00:57:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:42.938855+0000", 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:42.938856+0000", 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:42.938862+0000", 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6449999999999999e-06 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:42.938883+0000", 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0959999999999999e-05 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:42.938895+0000", 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.153e-05 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:42.939050+0000", 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015539500000000001 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:42.939113+0000", 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2966000000000007e-05 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:42.941441+0000", 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023281819999999998 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:42.941549+0000", 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107753 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:554 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 212992~16296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.227826+0000", 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.59392631200000001, 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0042547410000000003, 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 554 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.227826+0000", 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.227826+0000", 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.227829+0000", 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7929999999999999e-06 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.227900+0000", 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0430000000000002e-05 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.227943+0000", 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3378999999999999e-05 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.227965+0000", 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2079999999999999e-05 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.228137+0000", 2024-09-18T00:57:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172242 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.228356+0000", 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218338 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.228639+0000", 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00028335099999999999 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.231647+0000", 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030074569999999998 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.231766+0000", 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119753 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.231947+0000", 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018078199999999999 2024-09-18T00:57:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.232080+0000", 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013313600000000001 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:555 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 212992~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.228094+0000", 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.59365763800000004, 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049616189999999996, 2024-09-18T00:57:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 555 2024-09-18T00:57:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.228094+0000", 2024-09-18T00:57:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.228094+0000", 2024-09-18T00:57:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.228096+0000", 2024-09-18T00:57:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1270000000000001e-06 2024-09-18T00:57:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.228122+0000", 2024-09-18T00:57:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5316e-05 2024-09-18T00:57:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.228151+0000", 2024-09-18T00:57:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9127000000000001e-05 2024-09-18T00:57:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.228177+0000", 2024-09-18T00:57:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6557e-05 2024-09-18T00:57:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.229790+0000", 2024-09-18T00:57:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001612187 2024-09-18T00:57:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.230201+0000", 2024-09-18T00:57:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041179999999999998 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.230736+0000", 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00053481600000000002 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.232205+0000", 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014688419999999999 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.232827+0000", 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062191899999999997 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.232918+0000", 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1439000000000003e-05 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.233056+0000", 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137489 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4476.0:554 3.c e29/24 ondisk, result = 0)", 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.230096+0000", 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.59165544999999997, 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020045250000000001, 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:43.848 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.230096+0000", 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.230096+0000", 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.230098+0000", 2024-09-18T00:57:43.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9939999999999999e-06 2024-09-18T00:57:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.230111+0000", 2024-09-18T00:57:43.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2213e-05 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.230140+0000", 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8892999999999999e-05 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.230158+0000", 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712999999999999e-05 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.231729+0000", 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015708709999999999 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.231751+0000", 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1693000000000001e-05 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.232101+0000", 2024-09-18T00:57:43.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000350148 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:564 3.1b e29/24 3:dfaddc3a:::rbd_data.117c7eee262f.00000000000000c4:head v 29'84, mlcod=29'82)", 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.244245+0000", 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.577507039, 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002319719, 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.244245+0000", 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.244245+0000", 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.244247+0000", 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.663e-06 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.244266+0000", 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9017000000000001e-05 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.244295+0000", 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8996000000000002e-05 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.244307+0000", 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.271e-05 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.244454+0000", 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000146716 2024-09-18T00:57:43.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.244511+0000", 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7145999999999998e-05 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.246481+0000", 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019700080000000001 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.246565+0000", 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3462999999999997e-05 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:573 3.f 3:f3f8b533:::rbd_data.117c7eee262f.0000000000000140:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 245760~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.260673+0000", 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.56107861199999998, 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034487329999999998, 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 573 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.260673+0000", 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.260673+0000", 2024-09-18T00:57:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.260677+0000", 2024-09-18T00:57:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9729999999999999e-06 2024-09-18T00:57:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.260723+0000", 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6096e-05 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.260756+0000", 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2756000000000003e-05 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.260778+0000", 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1849999999999999e-05 2024-09-18T00:57:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.260925+0000", 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147529 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.261130+0000", 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020447799999999999 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:57:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.261327+0000", 2024-09-18T00:57:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019675199999999999 2024-09-18T00:57:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.262715+0000", 2024-09-18T00:57:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013880800000000001 2024-09-18T00:57:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.263953+0000", 2024-09-18T00:57:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012382070000000001 2024-09-18T00:57:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.264049+0000", 2024-09-18T00:57:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5599000000000004e-05 2024-09-18T00:57:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.264122+0000", 2024-09-18T00:57:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3412999999999997e-05 2024-09-18T00:57:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:577 3.c 3:33e20d71:::rbd_data.117c7eee262f.0000000000000144:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 237568~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.267404+0000", 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.55434823099999997, 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036722009999999999, 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 577 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.267404+0000", 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.267404+0000", 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.267407+0000", 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999999e-06 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.267454+0000", 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7012000000000003e-05 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.267496+0000", 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2676999999999998e-05 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.267518+0000", 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1203e-05 2024-09-18T00:57:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.267691+0000", 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017367899999999999 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.268209+0000", 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00051762200000000003 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-18T00:57:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.268649+0000", 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043985399999999999 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.270121+0000", 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001472768 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.270898+0000", 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077693100000000002 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.270999+0000", 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010043900000000001 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.271076+0000", 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6996000000000006e-05 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:584 3.c 3:31622561:::rbd_data.117c7eee262f.00000000000000dd:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 335872~8470] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.280983+0000", 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.54076890200000005, 2024-09-18T00:57:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035918149999999999, 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 584 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.280983+0000", 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.280983+0000", 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.280985+0000", 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6920000000000001e-06 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.281021+0000", 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6443000000000003e-05 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.281044+0000", 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2889000000000001e-05 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.281058+0000", 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3932e-05 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.281216+0000", 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015828500000000001 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.281606+0000", 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038951699999999998 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.282252+0000", 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064622099999999999 2024-09-18T00:57:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.284334+0000", 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002081861 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.284375+0000", 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1672999999999999e-05 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.284505+0000", 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000129913 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.284575+0000", 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9388999999999996e-05 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:586 3.1b e29/24 3:db7cab74:::rbd_data.117c7eee262f.00000000000000df:head v 29'85, mlcod=29'83)", 2024-09-18T00:57:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.285375+0000", 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.53637654000000001, 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017608039999999999, 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.285375+0000", 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.285375+0000", 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.285377+0000", 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.61e-06 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.285419+0000", 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.227e-05 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.285450+0000", 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1112999999999998e-05 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.285469+0000", 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8553999999999999e-05 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.285620+0000", 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015132200000000001 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.285682+0000", 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2019000000000004e-05 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.287044+0000", 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001361587 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.287136+0000", 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2329000000000003e-05 2024-09-18T00:57:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:589 3.c 3:31622561:::rbd_data.117c7eee262f.00000000000000dd:head [stat,write 458752~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.289143+0000", 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.53260920899999997, 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029553790000000002, 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 589 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289143+0000", 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289143+0000", 2024-09-18T00:57:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289146+0000", 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8830000000000002e-06 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289175+0000", 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9357e-05 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289209+0000", 2024-09-18T00:57:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4066999999999998e-05 2024-09-18T00:57:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289232+0000", 2024-09-18T00:57:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3238999999999999e-05 2024-09-18T00:57:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289363+0000", 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130455 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289490+0000", 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012683900000000001 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289665+0000", 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017558100000000001 2024-09-18T00:57:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.291813+0000", 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021474459999999999 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.291853+0000", 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0945000000000002e-05 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.291991+0000", 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137293 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.292098+0000", 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107274 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:591 3.1b e29/24 3:db7cab74:::rbd_data.117c7eee262f.00000000000000df:head v 29'86, mlcod=29'84)", 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.289747+0000", 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.53200519999999996, 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021990600000000001, 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289747+0000", 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289747+0000", 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289748+0000", 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5889999999999999e-06 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289799+0000", 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0906999999999999e-05 2024-09-18T00:57:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289825+0000", 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5845000000000001e-05 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289841+0000", 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5866999999999999e-05 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.289976+0000", 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013527 2024-09-18T00:57:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.290053+0000", 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7225999999999995e-05 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.291863+0000", 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001809237 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.291946+0000", 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3119000000000002e-05 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:590 3.1c e29/24 3:3c800619:::rbd_data.117c7eee262f.00000000000000de:head v 29'4, mlcod=29'2)", 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.290456+0000", 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.53129601599999998, 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021903069999999998, 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.290456+0000", 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.290456+0000", 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.290458+0000", 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8700000000000001e-06 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.290518+0000", 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0529999999999998e-05 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.290549+0000", 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0673000000000003e-05 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.290568+0000", 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8972999999999999e-05 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.290695+0000", 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012686699999999999 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.290738+0000", 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3489999999999999e-05 2024-09-18T00:57:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.292556+0000", 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001817431 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.292646+0000", 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0472999999999999e-05 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:609 3.3 3:c5251f9b:::rbd_data.117c7eee262f.0000000000000188:head [set-alloc-hint object_size 524288 write_size 524288,write 212992~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.318239+0000", 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.50351334800000003, 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033106419999999999, 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 609 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.318239+0000", 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.318239+0000", 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.318242+0000", 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4020000000000002e-06 2024-09-18T00:57:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.318265+0000", 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2591000000000001e-05 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.318294+0000", 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9904e-05 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.318311+0000", 2024-09-18T00:57:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6181e-05 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.318462+0000", 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015167499999999999 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.318812+0000", 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034968999999999999 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:57:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.319225+0000", 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041273699999999999 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.321294+0000", 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020692409999999999 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.321321+0000", 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7508000000000002e-05 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.321442+0000", 2024-09-18T00:57:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120105 2024-09-18T00:57:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.321549+0000", 2024-09-18T00:57:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107608 2024-09-18T00:57:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:615 3.3 3:c5251f9b:::rbd_data.117c7eee262f.0000000000000188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.329488+0000", 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.49226377399999999, 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024756399999999999, 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 615 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.329488+0000", 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.329488+0000", 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.329490+0000", 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3659999999999999e-06 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.329501+0000", 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0984e-05 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.329537+0000", 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5756999999999997e-05 2024-09-18T00:57:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.329550+0000", 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2668e-05 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.329736+0000", 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018637699999999999 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.329940+0000", 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020384999999999999 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.330154+0000", 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021371000000000001 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.331360+0000", 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00120627 2024-09-18T00:57:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.331876+0000", 2024-09-18T00:57:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00051586600000000002 2024-09-18T00:57:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.331918+0000", 2024-09-18T00:57:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1569999999999997e-05 2024-09-18T00:57:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.331964+0000", 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6221999999999999e-05 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:645 3.c 3:31622561:::rbd_data.117c7eee262f.00000000000000dd:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.370166+0000", 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.451586135, 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00260012, 2024-09-18T00:57:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 645 2024-09-18T00:57:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.370166+0000", 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.370166+0000", 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.370168+0000", 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5409999999999999e-06 2024-09-18T00:57:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.370178+0000", 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2990000000000002e-06 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.370205+0000", 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.370220+0000", 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5304000000000001e-05 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.370323+0000", 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103385 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.370447+0000", 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012351899999999999 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.370602+0000", 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000154683 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.372346+0000", 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001744055 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.372556+0000", 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021020300000000001 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.372671+0000", 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011552000000000001 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.372766+0000", 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4582999999999999e-05 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:646 3.1c e29/24 3:3c800619:::rbd_data.117c7eee262f.00000000000000de:head v 29'6, mlcod=29'4)", 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.371207+0000", 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.45054440099999998, 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002171255, 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371207+0000", 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371207+0000", 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371209+0000", 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371219+0000", 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0251999999999999e-05 2024-09-18T00:57:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371236+0000", 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6858999999999999e-05 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371248+0000", 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1833e-05 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371361+0000", 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112947 2024-09-18T00:57:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371392+0000", 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1337000000000002e-05 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373306+0000", 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019136229999999999 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373379+0000", 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2892000000000006e-05 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:640 3.1b e29/24 3:dfaddc3a:::rbd_data.117c7eee262f.00000000000000c4:head v 29'87, mlcod=29'85)", 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.371540+0000", 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.450212102, 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016863169999999999, 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371540+0000", 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371540+0000", 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371541+0000", 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371574+0000", 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3198000000000003e-05 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371600+0000", 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5432e-05 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371615+0000", 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5242000000000001e-05 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371704+0000", 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9098999999999995e-05 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.371727+0000", 2024-09-18T00:57:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2733000000000001e-05 2024-09-18T00:57:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373177+0000", 2024-09-18T00:57:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001450352 2024-09-18T00:57:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373226+0000", 2024-09-18T00:57:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8891000000000003e-05 2024-09-18T00:57:43.926 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.926 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.926 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:649 3.f 3:f3f8b533:::rbd_data.117c7eee262f.0000000000000140:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.372881+0000", 2024-09-18T00:57:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.44887074399999999, 2024-09-18T00:57:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019349460000000001, 2024-09-18T00:57:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 649 2024-09-18T00:57:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.372881+0000", 2024-09-18T00:57:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.372881+0000", 2024-09-18T00:57:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.372884+0000", 2024-09-18T00:57:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4370000000000001e-06 2024-09-18T00:57:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.372894+0000", 2024-09-18T00:57:43.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0643e-05 2024-09-18T00:57:43.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.372923+0000", 2024-09-18T00:57:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8589e-05 2024-09-18T00:57:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.372939+0000", 2024-09-18T00:57:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6098000000000001e-05 2024-09-18T00:57:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373025+0000", 2024-09-18T00:57:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6037000000000003e-05 2024-09-18T00:57:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373106+0000", 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1081999999999994e-05 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373287+0000", 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181428 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.374499+0000", 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001211441 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.374727+0000", 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022848200000000001 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.374772+0000", 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4836000000000002e-05 2024-09-18T00:57:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.374816+0000", 2024-09-18T00:57:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3872999999999998e-05 2024-09-18T00:57:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:653 3.c 3:33e20d71:::rbd_data.117c7eee262f.0000000000000144:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.373247+0000", 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.44850486899999997, 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018837719999999999, 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 653 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373247+0000", 2024-09-18T00:57:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373247+0000", 2024-09-18T00:57:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373249+0000", 2024-09-18T00:57:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7770000000000001e-06 2024-09-18T00:57:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373259+0000", 2024-09-18T00:57:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7850000000000003e-06 2024-09-18T00:57:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373286+0000", 2024-09-18T00:57:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.705e-05 2024-09-18T00:57:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373315+0000", 2024-09-18T00:57:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9663999999999999e-05 2024-09-18T00:57:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373437+0000", 2024-09-18T00:57:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121532 2024-09-18T00:57:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373597+0000", 2024-09-18T00:57:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016007099999999999 2024-09-18T00:57:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-18T00:57:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.373824+0000", 2024-09-18T00:57:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022662600000000001 2024-09-18T00:57:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.374599+0000", 2024-09-18T00:57:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000775539 2024-09-18T00:57:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.375046+0000", 2024-09-18T00:57:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00044705899999999998 2024-09-18T00:57:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.375097+0000", 2024-09-18T00:57:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0813000000000003e-05 2024-09-18T00:57:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.375131+0000", 2024-09-18T00:57:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3856000000000002e-05 2024-09-18T00:57:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:679 3.c 3:32eef993:::rbd_data.117c7eee262f.0000000000000066:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:57:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:43.396848+0000", 2024-09-18T00:57:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.424903584, 2024-09-18T00:57:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019726879999999998, 2024-09-18T00:57:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 679 2024-09-18T00:57:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.396848+0000", 2024-09-18T00:57:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.396848+0000", 2024-09-18T00:57:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.396851+0000", 2024-09-18T00:57:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3860000000000001e-06 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.396867+0000", 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6721000000000001e-05 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.396883+0000", 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5888000000000001e-05 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.396893+0000", 2024-09-18T00:57:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0169999999999999e-05 2024-09-18T00:57:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.396965+0000", 2024-09-18T00:57:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1897000000000006e-05 2024-09-18T00:57:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.397126+0000", 2024-09-18T00:57:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016070599999999999 2024-09-18T00:57:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-18T00:57:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.397268+0000", 2024-09-18T00:57:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014159599999999999 2024-09-18T00:57:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.397861+0000", 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059361600000000004 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.398710+0000", 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000848929 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.398772+0000", 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2039999999999996e-05 2024-09-18T00:57:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:43.398821+0000", 2024-09-18T00:57:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8739000000000002e-05 2024-09-18T00:57:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:43.952 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:43.953 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:57:43.954 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:43.950+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:43.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:57:44.055 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:44.050+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:44.065 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone1 from 0x5e71d08 (overlap 0x5e71d08) to 0x0 2024-09-18T00:57:44.156 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:44.150+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:44.160 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:44.160 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:44.160 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:44.160 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:44.182 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:57:44.270 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:44.266+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:44.358 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:44.354+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:44.382 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:44.383 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:44.408 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:57:44.456 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-18T00:57:44.458 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:44.454+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:44.560 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:44.554+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.291918155000001, 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:57:44.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:57:44.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:57:44.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:57:44.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.542140+0000", 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.039097927, 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012446464000000001, 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 34 2024-09-18T00:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542140+0000", 2024-09-18T00:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542140+0000", 2024-09-18T00:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542141+0000", 2024-09-18T00:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9400000000000005e-07 2024-09-18T00:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542156+0000", 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4786e-05 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542169+0000", 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3003000000000001e-05 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542177+0000", 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9929999999999995e-06 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.548827+0000", 2024-09-18T00:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066500989999999996 2024-09-18T00:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.549121+0000", 2024-09-18T00:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029445799999999999 2024-09-18T00:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.549517+0000", 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039595100000000002 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.554271+0000", 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047541959999999996 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.554311+0000", 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9756999999999999e-05 2024-09-18T00:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.554479+0000", 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016748500000000001 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.554587+0000", 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010804200000000001 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.542190+0000", 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.039047814, 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014197601000000001, 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 35 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542190+0000", 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542190+0000", 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542191+0000", 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1699999999999997e-07 2024-09-18T00:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542333+0000", 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000141762 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542351+0000", 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7796999999999999e-05 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542359+0000", 2024-09-18T00:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8899999999999996e-06 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.551099+0000", 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087392089999999995 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.551233+0000", 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013385900000000001 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.551444+0000", 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021182500000000001 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.554620+0000", 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031753649999999999 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.556091+0000", 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014711189999999999 2024-09-18T00:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.556251+0000", 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000159679 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.556388+0000", 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137379 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.542376+0000", 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.038862054999999, 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014268856, 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 36 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542376+0000", 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542376+0000", 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542377+0000", 2024-09-18T00:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1699999999999997e-07 2024-09-18T00:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542394+0000", 2024-09-18T00:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6988999999999999e-05 2024-09-18T00:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542405+0000", 2024-09-18T00:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0644e-05 2024-09-18T00:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542415+0000", 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0193e-05 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.552645+0000", 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010230288000000001 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.552895+0000", 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025024100000000001 2024-09-18T00:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.553327+0000", 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000431863 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.555943+0000", 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026161169999999998 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.556496+0000", 2024-09-18T00:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00055305199999999995 2024-09-18T00:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.556581+0000", 2024-09-18T00:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4573000000000005e-05 2024-09-18T00:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.556645+0000", 2024-09-18T00:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4078999999999994e-05 2024-09-18T00:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.542428+0000", 2024-09-18T00:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.038810286, 2024-09-18T00:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014847209, 2024-09-18T00:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 37 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542428+0000", 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542428+0000", 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542429+0000", 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1399999999999996e-07 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542435+0000", 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7560000000000002e-06 2024-09-18T00:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542445+0000", 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8230000000000006e-06 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.542453+0000", 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6639999999999998e-06 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.554704+0000", 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012250881999999999 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.554940+0000", 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000236562 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.555310+0000", 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036943700000000002 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.556670+0000", 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001360656 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.557142+0000", 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047134999999999999 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.557217+0000", 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5044999999999997e-05 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.557275+0000", 2024-09-18T00:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8319999999999997e-05 2024-09-18T00:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.648590+0000", 2024-09-18T00:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.932647857999999, 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013718584000000001, 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 86 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648590+0000", 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648590+0000", 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648591+0000", 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.06e-07 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648597+0000", 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648606+0000", 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2569999999999996e-06 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648618+0000", 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2013e-05 2024-09-18T00:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.655921+0000", 2024-09-18T00:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0073028670000000002 2024-09-18T00:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.656160+0000", 2024-09-18T00:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238695 2024-09-18T00:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.656543+0000", 2024-09-18T00:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000383197 2024-09-18T00:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.661791+0000", 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0052485759999999996 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.661980+0000", 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018876900000000001 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.662126+0000", 2024-09-18T00:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000146095 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.662309+0000", 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018262199999999999 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.648643+0000", 2024-09-18T00:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.932595161999998, 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013853890000000001, 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648643+0000", 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648643+0000", 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648644+0000", 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.540000000000001e-07 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648754+0000", 2024-09-18T00:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110215 2024-09-18T00:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648768+0000", 2024-09-18T00:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4073999999999999e-05 2024-09-18T00:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648777+0000", 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9099999999999994e-06 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.658068+0000", 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0092908650000000006 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.658310+0000", 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024164499999999999 2024-09-18T00:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.658686+0000", 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00037627299999999999 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.661938+0000", 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0032519749999999998 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.662332+0000", 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039408299999999999 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.662440+0000", 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010809 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.662497+0000", 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6805999999999998e-05 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.648795+0000", 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.932443472999999, 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014527408, 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 88 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648795+0000", 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648795+0000", 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648796+0000", 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7700000000000003e-07 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648802+0000", 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2929999999999997e-06 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648828+0000", 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6653e-05 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.648847+0000", 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.855e-05 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.660322+0000", 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011474598000000001 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.660577+0000", 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025570200000000001 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.660927+0000", 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034944700000000002 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.662514+0000", 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015874540000000001 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.663191+0000", 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00067720499999999995 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.663263+0000", 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1972999999999997e-05 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.663322+0000", 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8656000000000002e-05 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.821311+0000", 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.759927270999999, 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013912180999999999, 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 116 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821311+0000", 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821311+0000", 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821311+0000", 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9699999999999996e-07 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821315+0000", 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8890000000000004e-06 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821324+0000", 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6000000000000007e-06 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821328+0000", 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000003e-06 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.832709+0000", 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011380839 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.832840+0000", 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000130999 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.833057+0000", 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021682099999999999 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.834393+0000", 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001336297 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.835161+0000", 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00076809999999999997 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.835195+0000", 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4016000000000001e-05 2024-09-18T00:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.835223+0000", 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7813e-05 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.821352+0000", 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.759886487999999, 2024-09-18T00:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014625020000000001, 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 117 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821352+0000", 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821352+0000", 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821352+0000", 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-07 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821356+0000", 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1450000000000001e-06 2024-09-18T00:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821371+0000", 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4644000000000001e-05 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.821376+0000", 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7779999999999998e-06 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.833775+0000", 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012399652000000001 2024-09-18T00:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.833919+0000", 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000143745 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.834149+0000", 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230089 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.834998+0000", 2024-09-18T00:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084931999999999998 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.835815+0000", 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00081612299999999998 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.835911+0000", 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6280000000000007e-05 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.835977+0000", 2024-09-18T00:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5759000000000005e-05 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:24.319353+0000", 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.261885432, 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.011690607, 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319353+0000", 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319353+0000", 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319354+0000", 2024-09-18T00:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.773e-06 2024-09-18T00:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319368+0000", 2024-09-18T00:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3787e-05 2024-09-18T00:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319396+0000", 2024-09-18T00:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7840000000000001e-05 2024-09-18T00:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319421+0000", 2024-09-18T00:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4786999999999998e-05 2024-09-18T00:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.324910+0000", 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054888239999999998 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.325169+0000", 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025899400000000002 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.325552+0000", 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000383607 2024-09-18T00:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.329088+0000", 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035357829999999998 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.330905+0000", 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018168889999999999 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.330981+0000", 2024-09-18T00:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5647000000000003e-05 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.331043+0000", 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2675999999999994e-05 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:24.319488+0000", 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.261750379999999, 2024-09-18T00:57:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013352894000000001, 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 133 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319488+0000", 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319488+0000", 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319490+0000", 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319510+0000", 2024-09-18T00:57:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0380000000000001e-05 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319539+0000", 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8603999999999999e-05 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319562+0000", 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3298999999999998e-05 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.327339+0000", 2024-09-18T00:57:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077774599999999999 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.327663+0000", 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032324099999999999 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.327888+0000", 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022527800000000001 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.332685+0000", 2024-09-18T00:57:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047968339999999998 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.332706+0000", 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1483000000000001e-05 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.332779+0000", 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2953000000000004e-05 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.332841+0000", 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1482000000000005e-05 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:24.319598+0000", 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.261639766999998, 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013487860000000001, 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 134 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319598+0000", 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319598+0000", 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319600+0000", 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5030000000000001e-06 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319868+0000", 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026850099999999998 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319896+0000", 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7883999999999999e-05 2024-09-18T00:57:44.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319922+0000", 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5996e-05 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.329199+0000", 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0092765880000000005 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.329459+0000", 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025998800000000001 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.329804+0000", 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034527100000000002 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.332908+0000", 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031043500000000001 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.332927+0000", 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8493000000000001e-05 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.333016+0000", 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9549000000000006e-05 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.333086+0000", 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9737e-05 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:24.319962+0000", 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.261276073000001, 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016239268000000001, 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 135 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319962+0000", 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319962+0000", 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319964+0000", 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7099999999999999e-06 2024-09-18T00:57:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.319977+0000", 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3030000000000001e-05 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320004+0000", 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7123000000000001e-05 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320024+0000", 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9760000000000001e-05 2024-09-18T00:57:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.331178+0000", 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01115419 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.331408+0000", 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023061499999999999 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.331777+0000", 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000368864 2024-09-18T00:57:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.335912+0000", 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.004135049 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.336060+0000", 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147782 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.336132+0000", 2024-09-18T00:57:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2099000000000002e-05 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.336201+0000", 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9046000000000003e-05 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:24.320055+0000", 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.261183590000002, 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016283074000000002, 2024-09-18T00:57:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 136 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320055+0000", 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320055+0000", 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320056+0000", 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320068+0000", 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1877e-05 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320092+0000", 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4159999999999999e-05 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320108+0000", 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5376e-05 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.333213+0000", 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.013105393 2024-09-18T00:57:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.333375+0000", 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016207799999999999 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.333586+0000", 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021070600000000001 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.336040+0000", 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024542140000000001 2024-09-18T00:57:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.336214+0000", 2024-09-18T00:57:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000174045 2024-09-18T00:57:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.336286+0000", 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1589999999999997e-05 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.336338+0000", 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1928999999999997e-05 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:24.320135+0000", 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.261103271, 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018770407999999999, 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320135+0000", 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320135+0000", 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320136+0000", 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5430000000000001e-06 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320401+0000", 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026455800000000002 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320432+0000", 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0800000000000003e-05 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320450+0000", 2024-09-18T00:57:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8162999999999999e-05 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.334300+0000", 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.01384984 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.334434+0000", 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013467899999999999 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.334630+0000", 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019533199999999999 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.338620+0000", 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0039899330000000002 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.338773+0000", 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015344800000000001 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.338845+0000", 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1509999999999998e-05 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.338905+0000", 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0602e-05 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:24.320545+0000", 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.260693218, 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018491764000000001, 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 138 2024-09-18T00:57:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320545+0000", 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320545+0000", 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320547+0000", 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.747e-06 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320561+0000", 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3896999999999999e-05 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320585+0000", 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4300000000000001e-05 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320609+0000", 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.41e-05 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.336454+0000", 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015845007000000001 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.336658+0000", 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000203802 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.336859+0000", 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020141199999999999 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.338750+0000", 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001890512 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.338920+0000", 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017057500000000001 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.338985+0000", 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5176000000000001e-05 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.339037+0000", 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1236000000000002e-05 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-18T00:57:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:24.320644+0000", 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.260593794999998, 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021284146, 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 139 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320644+0000", 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320644+0000", 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320646+0000", 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6899999999999999e-06 2024-09-18T00:57:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320664+0000", 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7813e-05 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320686+0000", 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1756999999999999e-05 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320707+0000", 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1339999999999999e-05 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.337546+0000", 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016839549999999998 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.337682+0000", 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000135365 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.337892+0000", 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021061499999999999 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.341505+0000", 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036126420000000001 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.341634+0000", 2024-09-18T00:57:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012876200000000001 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.341797+0000", 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000162829 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.341928+0000", 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013178300000000001 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:24.320758+0000", 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.260479809, 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021425801000000001, 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 140 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320758+0000", 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320758+0000", 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320760+0000", 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.320999+0000", 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238679 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.321025+0000", 2024-09-18T00:57:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6346000000000002e-05 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.321045+0000", 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.963e-05 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.339154+0000", 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018109030000000002 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.339299+0000", 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014557900000000001 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.339493+0000", 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019376800000000001 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.341513+0000", 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002020034 2024-09-18T00:57:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.342036+0000", 2024-09-18T00:57:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052337599999999998 2024-09-18T00:57:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.342123+0000", 2024-09-18T00:57:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6643000000000004e-05 2024-09-18T00:57:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.342184+0000", 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1153000000000002e-05 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:24.321081+0000", 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.260157455000002, 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021581995999999999, 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 141 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.321081+0000", 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.321081+0000", 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.321082+0000", 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7630000000000001e-06 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.321103+0000", 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0126999999999999e-05 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.321130+0000", 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7223e-05 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.321144+0000", 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4256999999999999e-05 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.340575+0000", 2024-09-18T00:57:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.019431269000000001 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.340705+0000", 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000129379 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.340895+0000", 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018982600000000001 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.342336+0000", 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001441452 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.342505+0000", 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016897200000000001 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.342584+0000", 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9559000000000002e-05 2024-09-18T00:57:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:24.342663+0000", 2024-09-18T00:57:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8169000000000003e-05 2024-09-18T00:57:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:44.711 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:44.713 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:57:44.713 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:44.710+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:44.759 INFO:teuthology.orchestra.run.smithi042.stdout:178 read 0x327425c thru 0x327ce82 (0x8c27 bytes) 2024-09-18T00:57:44.768 INFO:teuthology.orchestra.run.smithi042.stdout:180 punch from 0x3cf366b to 0x3cf502e, (0x19c3 bytes) 2024-09-18T00:57:44.794 INFO:teuthology.orchestra.run.smithi042.stdout:181 writesame 0xb8ec614 thru 0xb8ee3d7 data_size 0x1e(0x1dc4 bytes) 2024-09-18T00:57:44.794 INFO:teuthology.orchestra.run.smithi042.stdout:182 write 0xa3bf54e thru 0xa3c97e5 (0xa298 bytes) 2024-09-18T00:57:44.805 INFO:teuthology.orchestra.run.smithi042.stdout:183 write 0xb87c755 thru 0xb8861de (0x9a8a bytes) 2024-09-18T00:57:44.813 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:44.806+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:44.814 INFO:teuthology.orchestra.run.smithi042.stdout:186 write 0x7b7a8be thru 0x7b85033 (0xa776 bytes) 2024-09-18T00:57:44.824 INFO:teuthology.orchestra.run.smithi042.stdout:187 write 0x8a42eea thru 0x8a51863 (0xe97a bytes) 2024-09-18T00:57:44.846 INFO:teuthology.orchestra.run.smithi042.stdout:188 trunc from 0xb8ee3d8 to 0x3878157 2024-09-18T00:57:44.850 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:57:44.900 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:44.900 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:44.900 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:44.900 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:44.913 INFO:teuthology.orchestra.run.smithi042.stdout:191 punch from 0x1ad4a05 to 0x1ad7eb8, (0x34b3 bytes) 2024-09-18T00:57:44.913 INFO:teuthology.orchestra.run.smithi042.stdout:192 punch from 0x21d5938 to 0x21dd47e, (0x7b46 bytes) 2024-09-18T00:57:44.913 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:44.913 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:44.910+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:44.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:57:44.932 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:57:44.940 INFO:teuthology.orchestra.run.smithi042.stdout:194 write 0xcebb485 thru 0xcec9a10 (0xe58c bytes) 2024-09-18T00:57:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:195 writesame 0x739d49c thru 0x73a2ef7 data_size 0x2(0x5a5c bytes) 2024-09-18T00:57:44.967 INFO:teuthology.orchestra.run.smithi042.stdout:196 read 0x7c0ef18 thru 0x7c15e8c (0x6f75 bytes) 2024-09-18T00:57:44.967 INFO:teuthology.orchestra.run.smithi042.stdout:197 read 0x61ffe33 thru 0x6203def (0x3fbd bytes) 2024-09-18T00:57:44.968 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:44.982 INFO:teuthology.orchestra.run.smithi042.stdout:199 trunc from 0xcec9a11 to 0x6d7aa2 2024-09-18T00:57:45.014 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:45.010+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:45.023 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:57:45.023 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:57:45.046 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:57:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:57:45.066 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:45.067 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:45.074 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:57:45.074 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:57:45.083 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:57:45.099 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:57:45.099 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout:200 trunc from 0x6d7aa2 to 0x61b67d8 2024-09-18T00:57:45.115 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:45.110+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout:201 read 0x5df91b1 thru 0x5dfa466 (0x12b6 bytes) 2024-09-18T00:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:57:45.119 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout:204 write 0x7853220 thru 0x7858f49 (0x5d2a bytes) 2024-09-18T00:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:57:45.138 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout:206 write 0xbcedf16 thru 0xbcf062d (0x2718 bytes) 2024-09-18T00:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout:208 write 0x86401d6 thru 0x8648c32 (0x8a5d bytes) 2024-09-18T00:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout:209 writesame 0x86a2e2e thru 0x86b2181 data_size 0x57(0xf354 bytes) 2024-09-18T00:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:57:45.160 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout:212 read 0xfa5fd6 thru 0xfac2ae (0x62d9 bytes) 2024-09-18T00:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout:213 write 0xca4e951 thru 0xca51007 (0x26b7 bytes) 2024-09-18T00:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout:214 write 0x56641bd thru 0x56687f2 (0x4636 bytes) 2024-09-18T00:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:57:45.181 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout:215 write 0x624376b thru 0x6244f75 (0x180b bytes) 2024-09-18T00:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout:216 write 0x5ec9589 thru 0x5ecc2fd (0x2d75 bytes) 2024-09-18T00:57:45.192 INFO:teuthology.orchestra.run.smithi042.stdout:217 write 0x891aa10 thru 0x892a09f (0xf690 bytes) 2024-09-18T00:57:45.197 INFO:teuthology.orchestra.run.smithi042.stdout:219 write 0x1510eec thru 0x1517ad9 (0x6bee bytes) 2024-09-18T00:57:45.201 INFO:teuthology.orchestra.run.smithi042.stdout:220 write 0x287497c thru 0x287c831 (0x7eb6 bytes) 2024-09-18T00:57:45.207 INFO:teuthology.orchestra.run.smithi042.stdout:221 write 0x1026d13 thru 0x1027361 (0x64f bytes) 2024-09-18T00:57:45.212 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:45.212 INFO:teuthology.orchestra.run.smithi042.stdout:223 punch from 0x4e323db to 0x4e37d0e, (0x5933 bytes) 2024-09-18T00:57:45.213 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:45.213 INFO:teuthology.orchestra.run.smithi042.stdout:226 punch from 0xe3e4e5 to 0xe43919, (0x5434 bytes) 2024-09-18T00:57:45.213 INFO:teuthology.orchestra.run.smithi042.stdout:229 read 0x5d55c9d thru 0x5d60874 (0xabd8 bytes) 2024-09-18T00:57:45.213 INFO:teuthology.orchestra.run.smithi042.stdout:231 write 0x824f483 thru 0x8259af2 (0xa670 bytes) 2024-09-18T00:57:45.215 INFO:teuthology.orchestra.run.smithi042.stdout:232 write 0x2201b75 thru 0x220b3dc (0x9868 bytes) 2024-09-18T00:57:45.216 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:45.210+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:45.222 INFO:teuthology.orchestra.run.smithi042.stdout:233 read 0x9455c39 thru 0x945773b (0x1b03 bytes) 2024-09-18T00:57:45.222 INFO:teuthology.orchestra.run.smithi042.stdout:234 write 0xa93df9f thru 0xa93e097 (0xf9 bytes) 2024-09-18T00:57:45.224 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:45.224 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:45.234 INFO:teuthology.orchestra.run.smithi042.stdout:235 trunc from 0xca51008 to 0x59f81ca 2024-09-18T00:57:45.245 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:57:45.255 INFO:teuthology.orchestra.run.smithi042.stdout:236 punch from 0x47f460 to 0x4838b2, (0x4452 bytes) 2024-09-18T00:57:45.258 INFO:teuthology.orchestra.run.smithi042.stdout:237 trunc from 0x59f81ca to 0x8b8ccd0 2024-09-18T00:57:45.277 INFO:teuthology.orchestra.run.smithi042.stdout:238 writesame 0xc9f30b0 thru 0xc9fbe7c data_size 0x1(0x8dcd bytes) 2024-09-18T00:57:45.277 INFO:teuthology.orchestra.run.smithi042.stdout:239 write 0x195ad36 thru 0x1960839 (0x5b04 bytes) 2024-09-18T00:57:45.284 INFO:teuthology.orchestra.run.smithi042.stdout:240 writesame 0xc5a362a thru 0xc5add3e data_size 0x1(0xa715 bytes) 2024-09-18T00:57:45.317 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:45.310+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:45.321 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:45.321 INFO:teuthology.orchestra.run.smithi042.stdout:244 punch from 0x9abaaf4 to 0x9abcef9, (0x2405 bytes) 2024-09-18T00:57:45.321 INFO:teuthology.orchestra.run.smithi042.stdout:245 write 0x3ca022c thru 0x3ca41f3 (0x3fc8 bytes) 2024-09-18T00:57:45.329 INFO:teuthology.orchestra.run.smithi042.stdout:246 trunc from 0xc9fbe7d to 0xc3140d4 2024-09-18T00:57:45.353 INFO:teuthology.orchestra.run.smithi042.stdout:248 read 0x2e44931 thru 0x2e51606 (0xccd6 bytes) 2024-09-18T00:57:45.353 INFO:teuthology.orchestra.run.smithi042.stdout:249 writesame 0x7c1485d thru 0x7c21bd0 data_size 0x4e(0xd374 bytes) 2024-09-18T00:57:45.366 INFO:teuthology.orchestra.run.smithi042.stdout:250 clone 3 order 20 su 32768 sc 2 2024-09-18T00:57:45.373 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:45.373 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:45.386 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:57:45.430 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:45.426+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:45.518 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:45.514+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:45.552 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:45.552 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:45.566 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:57:45.619 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:45.618+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:45.720 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:45.718+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:45.736 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:45.736 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:45.750 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:57:45.821 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:45.814+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:45.880 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:45.880 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:45.895 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:57:45.942 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:45.938+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:46.016 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:46.016 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:46.023 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:46.018+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:46.030 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:57:46.101 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone2 from 0xc3140d4 (overlap 0x6d7aa2) to 0x6287f8 2024-09-18T00:57:46.124 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:46.118+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:46.164 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:57:46.164 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:57:46.164 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:57:46.207 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:46.208 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:46.222 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:57:46.225 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:46.218+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:46.238 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-18T00:57:46.251 INFO:teuthology.orchestra.run.smithi042.stdout:252 read 0x9502764 thru 0x950ec4b (0xc4e8 bytes) 2024-09-18T00:57:46.260 INFO:teuthology.orchestra.run.smithi042.stdout:253 trunc from 0xc3140d4 to 0xacb4ea5 2024-09-18T00:57:46.280 INFO:teuthology.orchestra.run.smithi042.stdout:254 read 0x2baa32a thru 0x2bb1c11 (0x78e8 bytes) 2024-09-18T00:57:46.280 INFO:teuthology.orchestra.run.smithi042.stdout:255 write 0x309c968 thru 0x30ab58e (0xec27 bytes) 2024-09-18T00:57:46.291 INFO:teuthology.orchestra.run.smithi042.stdout:259 read 0x46f68c2 thru 0x46fe587 (0x7cc6 bytes) 2024-09-18T00:57:46.306 INFO:teuthology.orchestra.run.smithi042.stdout:260 write 0xce6286f thru 0xce62ca8 (0x43a bytes) 2024-09-18T00:57:46.310 INFO:teuthology.orchestra.run.smithi042.stdout:261 write 0x27efdf0 thru 0x27f5d09 (0x5f1a bytes) 2024-09-18T00:57:46.320 INFO:teuthology.orchestra.run.smithi042.stdout:262 writesame 0x9258133 thru 0x925dec2 data_size 0x18(0x5d90 bytes) 2024-09-18T00:57:46.325 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:46.318+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:46.326 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:46.326 INFO:teuthology.orchestra.run.smithi042.stdout:265 read 0x1a79e46 thru 0x1a86ffd (0xd1b8 bytes) 2024-09-18T00:57:46.328 INFO:teuthology.orchestra.run.smithi042.stdout:266 read 0xa50e6ed thru 0xa50e935 (0x249 bytes) 2024-09-18T00:57:46.328 INFO:teuthology.orchestra.run.smithi042.stdout:267 clone 4 order 23 su 32768 sc 15 2024-09-18T00:57:46.398 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:46.398 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:46.412 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:57:46.457 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:46.454+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:46.498 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:57:46.499 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:57:46.517 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-18T00:57:45.944038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":37731625,"num_objects":249,"num_object_clones":21,"num_object_copies":498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":249,"num_whiteouts":186,"num_read":6192,"num_read_kb":16635,"num_write":1062,"num_write_kb":6616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1059,"ondisk_log_size":1059,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274324,"kb_used_data":7960,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754700396,"statfs":{"total":773094113280,"available":772813205504,"internally_reserved":0,"allocated":8151040,"data_stored":6354116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33297813,"num_objects":238,"num_object_clones":21,"num_object_copies":476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":238,"num_whiteouts":186,"num_read":5916,"num_read_kb":16404,"num_write":876,"num_write_kb":4091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.031005"},"pg_stats":[{"pgid":"3.1f","version":"31'13","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.176717+0000","last_change":"2024-09-18T00:57:38.909849+0000","last_active":"2024-09-18T00:57:45.176717+0000","last_peered":"2024-09-18T00:57:45.176717+0000","last_clean":"2024-09-18T00:57:45.176717+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T00:57:45.176717+0000","last_undegraded":"2024-09-18T00:57:45.176717+0000","last_fullsized":"2024-09-18T00:57:45.176717+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:04:49.706386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001565786,"stat_sum":{"num_bytes":16,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":13,"num_read_kb":2,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"30'22","reported_seq":601,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.941129+0000","last_change":"2024-09-18T00:57:38.908434+0000","last_active":"2024-09-18T00:57:44.941129+0000","last_peered":"2024-09-18T00:57:44.941129+0000","last_clean":"2024-09-18T00:57:44.941129+0000","last_became_active":"2024-09-18T00:57:36.897948+0000","last_became_peered":"2024-09-18T00:57:36.897948+0000","last_unstale":"2024-09-18T00:57:44.941129+0000","last_undegraded":"2024-09-18T00:57:44.941129+0000","last_fullsized":"2024-09-18T00:57:44.941129+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:00:19.156603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040574399999999998,"stat_sum":{"num_bytes":2871424,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":10,"num_read":562,"num_read_kb":2985,"num_write":11,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"31'17","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.991241+0000","last_change":"2024-09-18T00:57:38.902294+0000","last_active":"2024-09-18T00:57:44.991241+0000","last_peered":"2024-09-18T00:57:44.991241+0000","last_clean":"2024-09-18T00:57:44.991241+0000","last_became_active":"2024-09-18T00:57:36.906790+0000","last_became_peered":"2024-09-18T00:57:36.906790+0000","last_unstale":"2024-09-18T00:57:44.991241+0000","last_undegraded":"2024-09-18T00:57:44.991241+0000","last_fullsized":"2024-09-18T00:57:44.991241+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:44.333416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039089,"stat_sum":{"num_bytes":1835008,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":6,"num_read_kb":10,"num_write":12,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"31'26","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.939400+0000","last_change":"2024-09-18T00:57:38.901600+0000","last_active":"2024-09-18T00:57:44.939400+0000","last_peered":"2024-09-18T00:57:44.939400+0000","last_clean":"2024-09-18T00:57:44.939400+0000","last_became_active":"2024-09-18T00:57:36.906003+0000","last_became_peered":"2024-09-18T00:57:36.906003+0000","last_unstale":"2024-09-18T00:57:44.939400+0000","last_undegraded":"2024-09-18T00:57:44.939400+0000","last_fullsized":"2024-09-18T00:57:44.939400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:19:52.085099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000711266,"stat_sum":{"num_bytes":1353738,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":26,"num_read_kb":14,"num_write":17,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"31'108","reported_seq":158,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.932831+0000","last_change":"2024-09-18T00:57:38.921854+0000","last_active":"2024-09-18T00:57:44.932831+0000","last_peered":"2024-09-18T00:57:44.932831+0000","last_clean":"2024-09-18T00:57:44.932831+0000","last_became_active":"2024-09-18T00:57:36.913393+0000","last_became_peered":"2024-09-18T00:57:36.913393+0000","last_unstale":"2024-09-18T00:57:44.932831+0000","last_undegraded":"2024-09-18T00:57:44.932831+0000","last_fullsized":"2024-09-18T00:57:44.932831+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:03:06.563787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002554022,"stat_sum":{"num_bytes":1160202,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":6,"num_read":424,"num_read_kb":310,"num_write":148,"num_write_kb":425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"31'60","reported_seq":441,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.352419+0000","last_change":"2024-09-18T00:57:38.901805+0000","last_active":"2024-09-18T00:57:45.352419+0000","last_peered":"2024-09-18T00:57:45.352419+0000","last_clean":"2024-09-18T00:57:45.352419+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T00:57:45.352419+0000","last_undegraded":"2024-09-18T00:57:45.352419+0000","last_fullsized":"2024-09-18T00:57:45.352419+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:58.724724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098070899999999996,"stat_sum":{"num_bytes":1835027,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":895,"num_read_kb":788,"num_write":51,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"30'13","reported_seq":739,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.075132+0000","last_change":"2024-09-18T00:57:38.922010+0000","last_active":"2024-09-18T00:57:45.075132+0000","last_peered":"2024-09-18T00:57:45.075132+0000","last_clean":"2024-09-18T00:57:45.075132+0000","last_became_active":"2024-09-18T00:57:36.910457+0000","last_became_peered":"2024-09-18T00:57:36.910457+0000","last_unstale":"2024-09-18T00:57:45.075132+0000","last_undegraded":"2024-09-18T00:57:45.075132+0000","last_fullsized":"2024-09-18T00:57:45.075132+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:58.257302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026842849999999998,"stat_sum":{"num_bytes":3025944,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":4,"num_read":708,"num_read_kb":4398,"num_write":9,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"30'40","reported_seq":673,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.941243+0000","last_change":"2024-09-18T00:57:38.908144+0000","last_active":"2024-09-18T00:57:44.941243+0000","last_peered":"2024-09-18T00:57:44.941243+0000","last_clean":"2024-09-18T00:57:44.941243+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T00:57:44.941243+0000","last_undegraded":"2024-09-18T00:57:44.941243+0000","last_fullsized":"2024-09-18T00:57:44.941243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:38:45.831518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000871597,"stat_sum":{"num_bytes":326784,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":6,"num_read":805,"num_read_kb":312,"num_write":31,"num_write_kb":439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"31'29","reported_seq":57,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.243014+0000","last_change":"2024-09-18T00:57:38.910038+0000","last_active":"2024-09-18T00:57:45.243014+0000","last_peered":"2024-09-18T00:57:45.243014+0000","last_clean":"2024-09-18T00:57:45.243014+0000","last_became_active":"2024-09-18T00:57:36.895915+0000","last_became_peered":"2024-09-18T00:57:36.895915+0000","last_unstale":"2024-09-18T00:57:45.243014+0000","last_undegraded":"2024-09-18T00:57:45.243014+0000","last_fullsized":"2024-09-18T00:57:45.243014+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:23.816896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001269769,"stat_sum":{"num_bytes":2071505,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":9,"num_read_kb":0,"num_write":22,"num_write_kb":378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"31'89","reported_seq":1146,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.214381+0000","last_change":"2024-09-18T00:57:39.381845+0000","last_active":"2024-09-18T00:57:45.214381+0000","last_peered":"2024-09-18T00:57:45.214381+0000","last_clean":"2024-09-18T00:57:45.214381+0000","last_became_active":"2024-09-18T00:57:36.894538+0000","last_became_peered":"2024-09-18T00:57:36.894538+0000","last_unstale":"2024-09-18T00:57:45.214381+0000","last_undegraded":"2024-09-18T00:57:45.214381+0000","last_fullsized":"2024-09-18T00:57:45.214381+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:07.278760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076552,"stat_sum":{"num_bytes":3330954,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":6,"num_read":1311,"num_read_kb":5937,"num_write":125,"num_write_kb":478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"31'22","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.299154+0000","last_change":"2024-09-18T00:57:38.909647+0000","last_active":"2024-09-18T00:57:45.299154+0000","last_peered":"2024-09-18T00:57:45.299154+0000","last_clean":"2024-09-18T00:57:45.299154+0000","last_became_active":"2024-09-18T00:57:36.902977+0000","last_became_peered":"2024-09-18T00:57:36.902977+0000","last_unstale":"2024-09-18T00:57:45.299154+0000","last_undegraded":"2024-09-18T00:57:45.299154+0000","last_fullsized":"2024-09-18T00:57:45.299154+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:28.976554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051351900000000004,"stat_sum":{"num_bytes":1867776,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":5,"num_read_kb":0,"num_write":17,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"31'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.197411+0000","last_change":"2024-09-18T00:57:38.902578+0000","last_active":"2024-09-18T00:57:45.197411+0000","last_peered":"2024-09-18T00:57:45.197411+0000","last_clean":"2024-09-18T00:57:45.197411+0000","last_became_active":"2024-09-18T00:57:36.894977+0000","last_became_peered":"2024-09-18T00:57:36.894977+0000","last_unstale":"2024-09-18T00:57:45.197411+0000","last_undegraded":"2024-09-18T00:57:45.197411+0000","last_fullsized":"2024-09-18T00:57:45.197411+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:02:26.522399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028223319999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.933971+0000","last_change":"2024-09-18T00:57:29.328163+0000","last_active":"2024-09-18T00:57:44.933971+0000","last_peered":"2024-09-18T00:57:44.933971+0000","last_clean":"2024-09-18T00:57:44.933971+0000","last_became_active":"2024-09-18T00:57:27.311231+0000","last_became_peered":"2024-09-18T00:57:27.311231+0000","last_unstale":"2024-09-18T00:57:44.933971+0000","last_undegraded":"2024-09-18T00:57:44.933971+0000","last_fullsized":"2024-09-18T00:57:44.933971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:57.340012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020990940000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"31'99","reported_seq":149,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.361382+0000","last_change":"2024-09-18T00:57:38.920098+0000","last_active":"2024-09-18T00:57:45.361382+0000","last_peered":"2024-09-18T00:57:45.361382+0000","last_clean":"2024-09-18T00:57:45.361382+0000","last_became_active":"2024-09-18T00:57:36.920599+0000","last_became_peered":"2024-09-18T00:57:36.920599+0000","last_unstale":"2024-09-18T00:57:45.361382+0000","last_undegraded":"2024-09-18T00:57:45.361382+0000","last_fullsized":"2024-09-18T00:57:45.361382+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:34.354929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276634,"stat_sum":{"num_bytes":59,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":482,"num_read_kb":370,"num_write":161,"num_write_kb":186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.933103+0000","last_change":"2024-09-18T00:57:29.327841+0000","last_active":"2024-09-18T00:57:44.933103+0000","last_peered":"2024-09-18T00:57:44.933103+0000","last_clean":"2024-09-18T00:57:44.933103+0000","last_became_active":"2024-09-18T00:57:27.315417+0000","last_became_peered":"2024-09-18T00:57:27.315417+0000","last_unstale":"2024-09-18T00:57:44.933103+0000","last_undegraded":"2024-09-18T00:57:44.933103+0000","last_fullsized":"2024-09-18T00:57:44.933103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:54.335334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"31'18","reported_seq":129,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.933021+0000","last_change":"2024-09-18T00:57:38.920634+0000","last_active":"2024-09-18T00:57:44.933021+0000","last_peered":"2024-09-18T00:57:44.933021+0000","last_clean":"2024-09-18T00:57:44.933021+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T00:57:44.933021+0000","last_undegraded":"2024-09-18T00:57:44.933021+0000","last_fullsized":"2024-09-18T00:57:44.933021+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:41:57.003454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728301,"stat_sum":{"num_bytes":229376,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":7,"num_read":94,"num_read_kb":224,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.933413+0000","last_change":"2024-09-18T00:57:29.327995+0000","last_active":"2024-09-18T00:57:44.933413+0000","last_peered":"2024-09-18T00:57:44.933413+0000","last_clean":"2024-09-18T00:57:44.933413+0000","last_became_active":"2024-09-18T00:57:27.316238+0000","last_became_peered":"2024-09-18T00:57:27.316238+0000","last_unstale":"2024-09-18T00:57:44.933413+0000","last_undegraded":"2024-09-18T00:57:44.933413+0000","last_fullsized":"2024-09-18T00:57:44.933413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019441149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"31'29","reported_seq":57,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.321004+0000","last_change":"2024-09-18T00:57:39.056519+0000","last_active":"2024-09-18T00:57:45.321004+0000","last_peered":"2024-09-18T00:57:45.321004+0000","last_clean":"2024-09-18T00:57:45.321004+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T00:57:45.321004+0000","last_undegraded":"2024-09-18T00:57:45.321004+0000","last_fullsized":"2024-09-18T00:57:45.321004+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:07:31.989391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043106399999999999,"stat_sum":{"num_bytes":1294034,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":9,"num_read":12,"num_read_kb":0,"num_write":20,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.941289+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:57:44.941289+0000","last_peered":"2024-09-18T00:57:44.941289+0000","last_clean":"2024-09-18T00:57:44.941289+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:57:44.941289+0000","last_undegraded":"2024-09-18T00:57:44.941289+0000","last_fullsized":"2024-09-18T00:57:44.941289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"31'25","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.241291+0000","last_change":"2024-09-18T00:57:39.381374+0000","last_active":"2024-09-18T00:57:45.241291+0000","last_peered":"2024-09-18T00:57:45.241291+0000","last_clean":"2024-09-18T00:57:45.241291+0000","last_became_active":"2024-09-18T00:57:36.902781+0000","last_became_peered":"2024-09-18T00:57:36.902781+0000","last_unstale":"2024-09-18T00:57:45.241291+0000","last_undegraded":"2024-09-18T00:57:45.241291+0000","last_fullsized":"2024-09-18T00:57:45.241291+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:56:36.580019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000584944,"stat_sum":{"num_bytes":2113536,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":9,"num_read":11,"num_read_kb":0,"num_write":16,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"31'18","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.148459+0000","last_change":"2024-09-18T00:57:38.909273+0000","last_active":"2024-09-18T00:57:45.148459+0000","last_peered":"2024-09-18T00:57:45.148459+0000","last_clean":"2024-09-18T00:57:45.148459+0000","last_became_active":"2024-09-18T00:57:36.906264+0000","last_became_peered":"2024-09-18T00:57:36.906264+0000","last_unstale":"2024-09-18T00:57:45.148459+0000","last_undegraded":"2024-09-18T00:57:45.148459+0000","last_fullsized":"2024-09-18T00:57:45.148459+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:57.706640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056105599999999999,"stat_sum":{"num_bytes":262144,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":6,"num_read_kb":0,"num_write":11,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.933376+0000","last_change":"2024-09-18T00:57:29.327436+0000","last_active":"2024-09-18T00:57:44.933376+0000","last_peered":"2024-09-18T00:57:44.933376+0000","last_clean":"2024-09-18T00:57:44.933376+0000","last_became_active":"2024-09-18T00:57:27.410600+0000","last_became_peered":"2024-09-18T00:57:27.410600+0000","last_unstale":"2024-09-18T00:57:44.933376+0000","last_undegraded":"2024-09-18T00:57:44.933376+0000","last_fullsized":"2024-09-18T00:57:44.933376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:59.380173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"31'18","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.941403+0000","last_change":"2024-09-18T00:57:38.907753+0000","last_active":"2024-09-18T00:57:44.941403+0000","last_peered":"2024-09-18T00:57:44.941403+0000","last_clean":"2024-09-18T00:57:44.941403+0000","last_became_active":"2024-09-18T00:57:36.907660+0000","last_became_peered":"2024-09-18T00:57:36.907660+0000","last_unstale":"2024-09-18T00:57:44.941403+0000","last_undegraded":"2024-09-18T00:57:44.941403+0000","last_fullsized":"2024-09-18T00:57:44.941403+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:24:14.127133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090282999999999997,"stat_sum":{"num_bytes":409616,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":16,"num_read_kb":2,"num_write":12,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T00:57:43.010312+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:57:43.010312+0000","last_peered":"2024-09-18T00:57:43.010312+0000","last_clean":"2024-09-18T00:57:43.010312+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:57:43.010312+0000","last_undegraded":"2024-09-18T00:57:43.010312+0000","last_fullsized":"2024-09-18T00:57:43.010312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"31'20","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.176338+0000","last_change":"2024-09-18T00:57:39.056604+0000","last_active":"2024-09-18T00:57:45.176338+0000","last_peered":"2024-09-18T00:57:45.176338+0000","last_clean":"2024-09-18T00:57:45.176338+0000","last_became_active":"2024-09-18T00:57:36.903081+0000","last_became_peered":"2024-09-18T00:57:36.903081+0000","last_unstale":"2024-09-18T00:57:45.176338+0000","last_undegraded":"2024-09-18T00:57:45.176338+0000","last_fullsized":"2024-09-18T00:57:45.176338+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:20.119913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045206800000000002,"stat_sum":{"num_bytes":1015808,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7,"num_read_kb":0,"num_write":14,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.933118+0000","last_change":"2024-09-18T00:57:29.327276+0000","last_active":"2024-09-18T00:57:44.933118+0000","last_peered":"2024-09-18T00:57:44.933118+0000","last_clean":"2024-09-18T00:57:44.933118+0000","last_became_active":"2024-09-18T00:57:27.316217+0000","last_became_peered":"2024-09-18T00:57:27.316217+0000","last_unstale":"2024-09-18T00:57:44.933118+0000","last_undegraded":"2024-09-18T00:57:44.933118+0000","last_fullsized":"2024-09-18T00:57:44.933118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:38.026720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012774469999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"31'18","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.951276+0000","last_change":"2024-09-18T00:57:38.919998+0000","last_active":"2024-09-18T00:57:44.951276+0000","last_peered":"2024-09-18T00:57:44.951276+0000","last_clean":"2024-09-18T00:57:44.951276+0000","last_became_active":"2024-09-18T00:57:36.906680+0000","last_became_peered":"2024-09-18T00:57:36.906680+0000","last_unstale":"2024-09-18T00:57:44.951276+0000","last_undegraded":"2024-09-18T00:57:44.951276+0000","last_fullsized":"2024-09-18T00:57:44.951276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:59:39.939782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064609,"stat_sum":{"num_bytes":1197097,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":14,"num_read_kb":53,"num_write":13,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.933568+0000","last_change":"2024-09-18T00:57:29.328733+0000","last_active":"2024-09-18T00:57:44.933568+0000","last_peered":"2024-09-18T00:57:44.933568+0000","last_clean":"2024-09-18T00:57:44.933568+0000","last_became_active":"2024-09-18T00:57:27.410396+0000","last_became_peered":"2024-09-18T00:57:27.410396+0000","last_unstale":"2024-09-18T00:57:44.933568+0000","last_undegraded":"2024-09-18T00:57:44.933568+0000","last_fullsized":"2024-09-18T00:57:44.933568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:27.438117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":186,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.939480+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:57:44.939480+0000","last_peered":"2024-09-18T00:57:44.939480+0000","last_clean":"2024-09-18T00:57:44.939480+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:57:44.939480+0000","last_undegraded":"2024-09-18T00:57:44.939480+0000","last_fullsized":"2024-09-18T00:57:44.939480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"31'17","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.162410+0000","last_change":"2024-09-18T00:57:38.919534+0000","last_active":"2024-09-18T00:57:45.162410+0000","last_peered":"2024-09-18T00:57:45.162410+0000","last_clean":"2024-09-18T00:57:45.162410+0000","last_became_active":"2024-09-18T00:57:36.908955+0000","last_became_peered":"2024-09-18T00:57:36.908955+0000","last_unstale":"2024-09-18T00:57:45.162410+0000","last_undegraded":"2024-09-18T00:57:45.162410+0000","last_fullsized":"2024-09-18T00:57:45.162410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:50:40.689072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077760500000000001,"stat_sum":{"num_bytes":1048576,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":5,"num_read_kb":0,"num_write":13,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"31'24","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.212734+0000","last_change":"2024-09-18T00:57:38.899513+0000","last_active":"2024-09-18T00:57:45.212734+0000","last_peered":"2024-09-18T00:57:45.212734+0000","last_clean":"2024-09-18T00:57:45.212734+0000","last_became_active":"2024-09-18T00:57:36.911337+0000","last_became_peered":"2024-09-18T00:57:36.911337+0000","last_unstale":"2024-09-18T00:57:45.212734+0000","last_undegraded":"2024-09-18T00:57:45.212734+0000","last_fullsized":"2024-09-18T00:57:45.212734+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:56:19.286486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061570699999999995,"stat_sum":{"num_bytes":1286144,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":10,"num_read_kb":0,"num_write":19,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"31'29","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.247522+0000","last_change":"2024-09-18T00:57:38.899904+0000","last_active":"2024-09-18T00:57:45.247522+0000","last_peered":"2024-09-18T00:57:45.247522+0000","last_clean":"2024-09-18T00:57:45.247522+0000","last_became_active":"2024-09-18T00:57:36.901837+0000","last_became_peered":"2024-09-18T00:57:36.901837+0000","last_unstale":"2024-09-18T00:57:45.247522+0000","last_undegraded":"2024-09-18T00:57:45.247522+0000","last_fullsized":"2024-09-18T00:57:45.247522+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:36:13.770356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097047300000000002,"stat_sum":{"num_bytes":2623546,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":12,"num_read":14,"num_read_kb":2,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"27'2","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T00:57:43.010414+0000","last_change":"2024-09-18T00:57:38.899763+0000","last_active":"2024-09-18T00:57:43.010414+0000","last_peered":"2024-09-18T00:57:43.010414+0000","last_clean":"2024-09-18T00:57:43.010414+0000","last_became_active":"2024-09-18T00:57:36.905869+0000","last_became_peered":"2024-09-18T00:57:36.905869+0000","last_unstale":"2024-09-18T00:57:43.010414+0000","last_undegraded":"2024-09-18T00:57:43.010414+0000","last_fullsized":"2024-09-18T00:57:43.010414+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:48:04.319723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097615600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"30'18","reported_seq":193,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.933765+0000","last_change":"2024-09-18T00:57:38.909165+0000","last_active":"2024-09-18T00:57:44.933765+0000","last_peered":"2024-09-18T00:57:44.933765+0000","last_clean":"2024-09-18T00:57:44.933765+0000","last_became_active":"2024-09-18T00:57:36.908288+0000","last_became_peered":"2024-09-18T00:57:36.908288+0000","last_unstale":"2024-09-18T00:57:44.933765+0000","last_undegraded":"2024-09-18T00:57:44.933765+0000","last_fullsized":"2024-09-18T00:57:44.933765+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:56:03.655326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044375399999999998,"stat_sum":{"num_bytes":450472,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":5,"num_read":160,"num_read_kb":440,"num_write":12,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"31'40","reported_seq":165,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.368030+0000","last_change":"2024-09-18T00:57:38.903046+0000","last_active":"2024-09-18T00:57:45.368030+0000","last_peered":"2024-09-18T00:57:45.368030+0000","last_clean":"2024-09-18T00:57:45.368030+0000","last_became_active":"2024-09-18T00:57:36.900286+0000","last_became_peered":"2024-09-18T00:57:36.900286+0000","last_unstale":"2024-09-18T00:57:45.368030+0000","last_undegraded":"2024-09-18T00:57:45.368030+0000","last_fullsized":"2024-09-18T00:57:45.368030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:00.629385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023201229999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":271,"num_read_kb":225,"num_write":32,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"30'14","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.950749+0000","last_change":"2024-09-18T00:57:39.382152+0000","last_active":"2024-09-18T00:57:44.950749+0000","last_peered":"2024-09-18T00:57:44.950749+0000","last_clean":"2024-09-18T00:57:44.950749+0000","last_became_active":"2024-09-18T00:57:36.904791+0000","last_became_peered":"2024-09-18T00:57:36.904791+0000","last_unstale":"2024-09-18T00:57:44.950749+0000","last_undegraded":"2024-09-18T00:57:44.950749+0000","last_fullsized":"2024-09-18T00:57:44.950749+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:26:15.066006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090629000000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5,"num_read_kb":0,"num_write":9,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"31'24","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.186898+0000","last_change":"2024-09-18T00:57:38.909597+0000","last_active":"2024-09-18T00:57:45.186898+0000","last_peered":"2024-09-18T00:57:45.186898+0000","last_clean":"2024-09-18T00:57:45.186898+0000","last_became_active":"2024-09-18T00:57:36.905462+0000","last_became_peered":"2024-09-18T00:57:36.905462+0000","last_unstale":"2024-09-18T00:57:45.186898+0000","last_undegraded":"2024-09-18T00:57:45.186898+0000","last_fullsized":"2024-09-18T00:57:45.186898+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:07.524103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081831799999999995,"stat_sum":{"num_bytes":82,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":46,"num_read_kb":31,"num_write":28,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"31'20","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.212379+0000","last_change":"2024-09-18T00:57:38.900906+0000","last_active":"2024-09-18T00:57:45.212379+0000","last_peered":"2024-09-18T00:57:45.212379+0000","last_clean":"2024-09-18T00:57:45.212379+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T00:57:45.212379+0000","last_undegraded":"2024-09-18T00:57:45.212379+0000","last_fullsized":"2024-09-18T00:57:45.212379+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:36:32.932733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069386899999999995,"stat_sum":{"num_bytes":3337788,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":9,"num_read_kb":0,"num_write":12,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"30'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:44.933256+0000","last_change":"2024-09-18T00:57:38.899297+0000","last_active":"2024-09-18T00:57:44.933256+0000","last_peered":"2024-09-18T00:57:44.933256+0000","last_clean":"2024-09-18T00:57:44.933256+0000","last_became_active":"2024-09-18T00:57:36.897289+0000","last_became_peered":"2024-09-18T00:57:36.897289+0000","last_unstale":"2024-09-18T00:57:44.933256+0000","last_undegraded":"2024-09-18T00:57:44.933256+0000","last_fullsized":"2024-09-18T00:57:44.933256+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:35:38.990105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071487799999999995,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3,"num_read_kb":0,"num_write":7,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"31'24","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.044419+0000","last_change":"2024-09-18T00:57:39.056761+0000","last_active":"2024-09-18T00:57:45.044419+0000","last_peered":"2024-09-18T00:57:45.044419+0000","last_clean":"2024-09-18T00:57:45.044419+0000","last_became_active":"2024-09-18T00:57:36.910991+0000","last_became_peered":"2024-09-18T00:57:36.910991+0000","last_unstale":"2024-09-18T00:57:45.044419+0000","last_undegraded":"2024-09-18T00:57:45.044419+0000","last_fullsized":"2024-09-18T00:57:45.044419+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:25.533851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055104700000000004,"stat_sum":{"num_bytes":1752214,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":8,"num_read_kb":0,"num_write":18,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"31'25","reported_seq":180,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-18T00:57:45.226380+0000","last_change":"2024-09-18T00:57:38.899783+0000","last_active":"2024-09-18T00:57:45.226380+0000","last_peered":"2024-09-18T00:57:45.226380+0000","last_clean":"2024-09-18T00:57:45.226380+0000","last_became_active":"2024-09-18T00:57:36.904999+0000","last_became_peered":"2024-09-18T00:57:36.904999+0000","last_unstale":"2024-09-18T00:57:45.226380+0000","last_undegraded":"2024-09-18T00:57:45.226380+0000","last_fullsized":"2024-09-18T00:57:45.226380+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:25:57.839227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070399799999999995,"stat_sum":{"num_bytes":442368,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":7,"num_read":137,"num_read_kb":432,"num_write":18,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":37141238,"num_objects":245,"num_object_clones":21,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":186,"num_read":6076,"num_read_kb":16535,"num_write":901,"num_write_kb":4198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4489216,"data_stored":4302738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":939,"ondisk_log_size":939,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1687552,"data_stored":1463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36904,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334936,"statfs":{"total":96636764160,"available":96598974464,"internally_reserved":0,"allocated":1589248,"data_stored":1346992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":745472,"data_stored":510740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35952,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335888,"statfs":{"total":96636764160,"available":96599949312,"internally_reserved":0,"allocated":618496,"data_stored":406345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":724992,"data_stored":499367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1236992,"data_stored":996161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1241088,"data_stored":1015141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35652,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336188,"statfs":{"total":96636764160,"available":96600256512,"internally_reserved":0,"allocated":307200,"data_stored":115828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":909312,"data_stored":887373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":341340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":371599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":354855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":358472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":628856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1351680,"data_stored":1335755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:57:46.536 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:46.536 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:46.550 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:57:46.562 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:46.558+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:46.638 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:46.634+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:46.715 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:46.715 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:46.729 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:57:46.730 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:46.726+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:46.831 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:46.826+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:46.898 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:46.898 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:46.915 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:57:46.933 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:46.926+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:46.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:57:47.034 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:47.030+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:47.037 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:47.037 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:47.040 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone3 from 0xce62ca9 (overlap 0xacb4ea5) to 0x565a752 2024-09-18T00:57:47.055 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:57:47.135 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:47.130+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:47.190 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:47.190 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:47.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:57:47.204 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:57:47.246 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:47.242+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:47.260 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-18T00:57:47.284 INFO:teuthology.orchestra.run.smithi042.stdout:268 read 0xc8ffa3f thru 0xc905276 (0x5838 bytes) 2024-09-18T00:57:47.289 INFO:teuthology.orchestra.run.smithi042.stdout:269 read 0x51e00a9 thru 0x51e1a5c (0x19b4 bytes) 2024-09-18T00:57:47.291 INFO:teuthology.orchestra.run.smithi042.stdout:270 clone 5 order 25 su 1048576 sc 2 2024-09-18T00:57:47.337 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:47.330+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:47.339 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:47.339 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:47.352 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:57:47.352 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:57:47.354 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:57:47.369 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:57:47.369 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:57:47.385 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:57:47.385 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:57:47.401 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:57:47.401 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:57:47.420 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:57:47.420 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:57:47.437 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:47.434+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:47.439 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:57:47.440 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:57:47.460 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:57:47.460 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:57:47.479 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:47.479 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:57:47.479 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:57:47.479 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:57:47.479 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:47.479 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:57:47.480 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:57:47.493 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:57:47.538 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:47.534+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:47.639 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:47.634+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:47.663 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:47.664 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:57:47.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:57:47.664 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:57:47.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.664 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4003 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.664 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.002322+0000", 2024-09-18T00:57:47.664 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6577308589999999, 2024-09-18T00:57:47.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033174606000000002, 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4003 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002322+0000", 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002322+0000", 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002324+0000", 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4100000000000001e-06 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002329+0000", 2024-09-18T00:57:47.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6130000000000003e-06 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002349+0000", 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9916999999999999e-05 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002365+0000", 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5803000000000001e-05 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.035093+0000", 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032728023000000002 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.035232+0000", 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013901300000000001 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.035497+0000", 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026482699999999998 2024-09-18T00:57:47.668 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.669 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.669 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.669 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4005 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.669 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.002387+0000", 2024-09-18T00:57:47.669 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6576661230000003, 2024-09-18T00:57:47.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033684370999999998, 2024-09-18T00:57:47.669 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.669 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.669 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.670 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.670 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.670 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.670 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4005 2024-09-18T00:57:47.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.670 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002387+0000", 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002387+0000", 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002388+0000", 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002398+0000", 2024-09-18T00:57:47.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4059999999999994e-06 2024-09-18T00:57:47.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002418+0000", 2024-09-18T00:57:47.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0186999999999998e-05 2024-09-18T00:57:47.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002441+0000", 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3332999999999998e-05 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.035594+0000", 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033153056 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.035713+0000", 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118608 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.036071+0000", 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035835400000000002 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4007 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.002483+0000", 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6575704140000003, 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.034092442000000001, 2024-09-18T00:57:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4007 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002483+0000", 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002483+0000", 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002484+0000", 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.274e-06 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002490+0000", 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3029999999999996e-06 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002513+0000", 2024-09-18T00:57:47.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2297e-05 2024-09-18T00:57:47.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002530+0000", 2024-09-18T00:57:47.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7563000000000001e-05 2024-09-18T00:57:47.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.036156+0000", 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033625836999999999 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.036259+0000", 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000103399 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.036575+0000", 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031576899999999998 2024-09-18T00:57:47.678 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4010 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.002553+0000", 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6575006180000003, 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.034492190999999998, 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.679 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4010 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002553+0000", 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002553+0000", 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002554+0000", 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.127e-06 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002560+0000", 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.55e-06 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002577+0000", 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6840000000000001e-05 2024-09-18T00:57:47.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002593+0000", 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6056e-05 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.036650+0000", 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.034056639 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.036765+0000", 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011528599999999999 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.037045+0000", 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027969300000000002 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4012 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.002629+0000", 2024-09-18T00:57:47.683 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.657423911, 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.034862854999999998, 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4012 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002629+0000", 2024-09-18T00:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002629+0000", 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002630+0000", 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.063e-06 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002637+0000", 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3230000000000003e-06 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002657+0000", 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0203999999999998e-05 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002672+0000", 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.525e-05 2024-09-18T00:57:47.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.037126+0000", 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.034453785000000001 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.037241+0000", 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011476600000000001 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.037492+0000", 2024-09-18T00:57:47.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025146400000000002 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4014 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.002697+0000", 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6573559219999998, 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.035276126999999997, 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.688 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4014 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002697+0000", 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002697+0000", 2024-09-18T00:57:47.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002698+0000", 2024-09-18T00:57:47.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1570000000000001e-06 2024-09-18T00:57:47.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002704+0000", 2024-09-18T00:57:47.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1e-06 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002724+0000", 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9919e-05 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002739+0000", 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4567e-05 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.037586+0000", 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.034847278000000002 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.037728+0000", 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014139999999999999 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.037973+0000", 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000245706 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.692 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.693 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4017 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.693 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.002759+0000", 2024-09-18T00:57:47.693 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6572943090000001, 2024-09-18T00:57:47.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.035678192999999997, 2024-09-18T00:57:47.693 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.693 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.693 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.693 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4017 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002759+0000", 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002759+0000", 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002760+0000", 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0529999999999999e-06 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002781+0000", 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0970000000000001e-05 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002801+0000", 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9939999999999999e-05 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002815+0000", 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3987e-05 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.038048+0000", 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.035233078000000001 2024-09-18T00:57:47.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.038192+0000", 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014397800000000001 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.038437+0000", 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024518700000000001 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4018 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.002867+0000", 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6571858969999997, 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.035976406000000002, 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4018 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002867+0000", 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002867+0000", 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002869+0000", 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.567e-06 2024-09-18T00:57:47.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002879+0000", 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.041e-05 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002901+0000", 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1967000000000002e-05 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002917+0000", 2024-09-18T00:57:47.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5639999999999999e-05 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.038476+0000", 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.035559009000000003 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.038594+0000", 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011793100000000001 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.038844+0000", 2024-09-18T00:57:47.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024988199999999999 2024-09-18T00:57:47.702 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.702 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.702 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.702 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4020 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 294912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.702 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.002943+0000", 2024-09-18T00:57:47.702 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6571101869999998, 2024-09-18T00:57:47.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036402279000000003, 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4020 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002943+0000", 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002943+0000", 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002944+0000", 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.407e-06 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002950+0000", 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9070000000000004e-06 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002969+0000", 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8796000000000001e-05 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.002983+0000", 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4406999999999999e-05 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.038928+0000", 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.035945044000000002 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039111+0000", 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000182368 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039345+0000", 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023435 2024-09-18T00:57:47.706 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.707 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.707 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.707 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4041 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.707 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:57:47.707 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6570491409999999, 2024-09-18T00:57:47.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037088000000000003, 2024-09-18T00:57:47.707 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4041 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003005+0000", 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.187e-06 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003011+0000", 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003029+0000", 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8026999999999999e-05 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003043+0000", 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039484+0000", 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036440895000000001 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039722+0000", 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000238507 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040092+0000", 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000369818 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.711 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6569915179999999, 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.712 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:57:47.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.716 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6569332609999998, 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:57:47.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:57:47.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6568748219999998, 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:57:47.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.725 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.656813605, 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.726 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:57:47.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:57:47.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:57:47.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:57:47.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:57:47.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:57:47.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:57:47.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:57:47.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:57:47.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6567231900000001, 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.732 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:57:47.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:57:47.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:57:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6566522570000002, 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:57:47.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.656592651, 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.741 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:57:47.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:57:47.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:57:47.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:57:47.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6565332809999997, 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.746 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:57:47.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:57:47.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:57:47.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:57:47.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6564714920000001, 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.751 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.755 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.6563386690000002, 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:57:47.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:47.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:47.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:57:47.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:57:47.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.760 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:47.760 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:47.760 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:47.761 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:57:47.762 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:47.758+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:47.863 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:47.858+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:47.899 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:47.899 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:47.900 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:47.900 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:47.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:57:47.952 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:57:47.963 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:47.958+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:48.046 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:48.046 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:48.046 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:48.046 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:48.046 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:48.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:57:48.063 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:48.058+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:48.105 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone4 from 0xce62ca9 (overlap 0xce62ca9) to 0x90785a9 2024-09-18T00:57:48.164 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:48.158+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:48.208 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:48.208 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:57:48.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:57:48.209 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:48.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.209 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3681 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.209 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867949+0000", 2024-09-18T00:57:48.209 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.337943036, 2024-09-18T00:57:48.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024162787000000002, 2024-09-18T00:57:48.209 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3681 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867949+0000", 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867949+0000", 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867949+0000", 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.65e-07 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867952+0000", 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8710000000000001e-06 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867958+0000", 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2759999999999997e-06 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867963+0000", 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6190000000000002e-06 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.891753+0000", 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023790430000000001 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892004+0000", 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000250967 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892112+0000", 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107259 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3710 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3379231649999999, 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024498702000000001, 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3710 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:57:48.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867971+0000", 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.671e-06 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867976+0000", 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7030000000000003e-06 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867981+0000", 2024-09-18T00:57:48.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8520000000000003e-06 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892175+0000", 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024193538000000001 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892279+0000", 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104017 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892467+0000", 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018858699999999999 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3711 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3379048450000002, 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024701471999999999, 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3711 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6199999999999999e-07 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867989+0000", 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867995+0000", 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.127e-06 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868000+0000", 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892500+0000", 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024499725999999999 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892587+0000", 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7232999999999996e-05 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892688+0000", 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010115000000000001 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3712 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3378858870000001, 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024914385000000001, 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3712 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.223 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868008+0000", 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868014+0000", 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868019+0000", 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892727+0000", 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024707638000000001 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892817+0000", 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9569999999999998e-05 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892920+0000", 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103744 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3713 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3378663560000001, 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025115858000000001, 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3713 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868026+0000", 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.58e-07 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868032+0000", 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3790000000000003e-06 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868039+0000", 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6969999999999996e-06 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868044+0000", 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7060000000000003e-06 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892954+0000", 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024910704999999998 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893041+0000", 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6747000000000001e-05 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893141+0000", 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010026600000000001 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3714 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3378415490000002, 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025291557999999999, 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3714 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868051+0000", 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868052+0000", 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.863e-06 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868059+0000", 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2720000000000003e-06 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868064+0000", 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8860000000000003e-06 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893175+0000", 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025111114 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893262+0000", 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7476000000000001e-05 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893342+0000", 2024-09-18T00:57:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9577000000000001e-05 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3715 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3378220440000002, 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025471191000000001, 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3715 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:48.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2799999999999996e-07 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868072+0000", 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9269999999999999e-06 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868078+0000", 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9830000000000001e-06 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868083+0000", 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.972e-06 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893373+0000", 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025290316 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893463+0000", 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9078000000000003e-05 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893541+0000", 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8386999999999993e-05 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.240 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3716 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3378029859999998, 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025664781000000001, 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3716 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2099999999999998e-07 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868091+0000", 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8950000000000001e-06 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868097+0000", 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1929999999999998e-06 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868102+0000", 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6789999999999996e-06 2024-09-18T00:57:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893570+0000", 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025468227999999999 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893657+0000", 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7176999999999996e-05 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893754+0000", 2024-09-18T00:57:48.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6287999999999997e-05 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3717 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3377838799999999, 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025815932999999999, 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3717 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868111+0000", 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6860000000000002e-06 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868117+0000", 2024-09-18T00:57:48.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868122+0000", 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.69e-06 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893802+0000", 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025680397000000001 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893869+0000", 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6597e-05 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893924+0000", 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5031999999999998e-05 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3718 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:48.249 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3377628580000001, 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026018052, 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3718 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:48.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868131+0000", 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9369999999999998e-06 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868138+0000", 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4389999999999997e-06 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868143+0000", 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1440000000000002e-06 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893973+0000", 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025830490000000001 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894063+0000", 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9313000000000004e-05 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894147+0000", 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4342e-05 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3719 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.337742902, 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026147934000000001, 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3719 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868151+0000", 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868157+0000", 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:57:48.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868162+0000", 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5619999999999997e-06 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894183+0000", 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026021083 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894241+0000", 2024-09-18T00:57:48.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8505999999999999e-05 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894297+0000", 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5396000000000003e-05 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3720 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:57:48.258 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.337724385, 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026376077000000001, 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3720 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:57:48.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868168+0000", 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868170+0000", 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0889999999999998e-06 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868176+0000", 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3090000000000001e-06 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868181+0000", 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8810000000000004e-06 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894341+0000", 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026159636 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894430+0000", 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8986000000000005e-05 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894543+0000", 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011382700000000001 2024-09-18T00:57:48.262 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3721 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3377048670000002, 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569722, 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3721 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868189+0000", 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7460000000000001e-06 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868195+0000", 2024-09-18T00:57:48.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8490000000000002e-06 2024-09-18T00:57:48.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868200+0000", 2024-09-18T00:57:48.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1669999999999998e-06 2024-09-18T00:57:48.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894623+0000", 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02642338 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894688+0000", 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4183000000000005e-05 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894757+0000", 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9062999999999999e-05 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3722 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3376861820000001, 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026735041000000001, 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3722 2024-09-18T00:57:48.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.269 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:48.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868208+0000", 2024-09-18T00:57:48.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8470000000000001e-06 2024-09-18T00:57:48.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868214+0000", 2024-09-18T00:57:48.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1090000000000003e-06 2024-09-18T00:57:48.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868218+0000", 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.476e-06 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894788+0000", 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569274 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894868+0000", 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0711999999999996e-05 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894941+0000", 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3723 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:48.273 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.337667766, 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026971922999999998, 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3723 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:48.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-07 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868226+0000", 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.812e-06 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868232+0000", 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7910000000000001e-06 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868237+0000", 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7339999999999999e-06 2024-09-18T00:57:48.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895056+0000", 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026819195000000001 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895103+0000", 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7264e-05 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895196+0000", 2024-09-18T00:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2802999999999999e-05 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3724 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.337649554, 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027181833999999998, 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.278 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3724 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868243+0000", 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868245+0000", 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1160000000000002e-06 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868251+0000", 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1410000000000002e-06 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868256+0000", 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0130000000000001e-06 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895249+0000", 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02699352 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895338+0000", 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8331000000000005e-05 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895424+0000", 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6348999999999996e-05 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3725 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.33763042, 2024-09-18T00:57:48.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027296051000000002, 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3725 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868262+0000", 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868267+0000", 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868274+0000", 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2360000000000001e-06 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868278+0000", 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7620000000000003e-06 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895458+0000", 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027179608000000001 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895511+0000", 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2738000000000002e-05 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895557+0000", 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6631999999999997e-05 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.286 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3726 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871463+0000", 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3344286049999998, 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024295104000000001, 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3726 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871463+0000", 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871463+0000", 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871466+0000", 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8169999999999999e-06 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871480+0000", 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3645999999999999e-05 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871506+0000", 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.637e-05 2024-09-18T00:57:48.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871535+0000", 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9249999999999999e-05 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895596+0000", 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024060537999999999 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895678+0000", 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2589999999999994e-05 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895758+0000", 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9893000000000002e-05 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3727 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3343198689999998, 2024-09-18T00:57:48.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024398075000000002, 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3727 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:57:48.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871573+0000", 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.367e-06 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871580+0000", 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8029999999999999e-06 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871600+0000", 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0213000000000001e-05 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871615+0000", 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.471e-05 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895805+0000", 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024189677 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895923+0000", 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118496 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895970+0000", 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6808999999999999e-05 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.295 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3728 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.33425861, 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024637071999999999, 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3728 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871634+0000", 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.074e-06 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871642+0000", 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8830000000000006e-06 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871660+0000", 2024-09-18T00:57:48.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8283000000000001e-05 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871692+0000", 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1714000000000002e-05 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896004+0000", 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024312099 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896099+0000", 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5036000000000003e-05 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896270+0000", 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017098300000000001 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.300 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:48.302 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:57:48.302 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:48.298+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-18T00:57:48.403 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:48.398+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:48.504 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:48.498+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:48.520 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:48.520 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:48.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7235 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484145+0000", 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022356088999999999, 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022380160999999999, 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7235 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484145+0000", 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484145+0000", 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484145+0000", 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-18T00:57:48.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484147+0000", 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9879999999999999e-06 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484154+0000", 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4829999999999996e-06 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484159+0000", 2024-09-18T00:57:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2959999999999998e-06 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7303 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484874+0000", 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021627401000000001, 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021728284, 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7303 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484874+0000", 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484874+0000", 2024-09-18T00:57:48.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484874+0000", 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484876+0000", 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.542e-06 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484883+0000", 2024-09-18T00:57:48.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2609999999999999e-06 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484888+0000", 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.882e-06 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7422 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485591+0000", 2024-09-18T00:57:48.527 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020909559000000001, 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021073537, 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7422 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485591+0000", 2024-09-18T00:57:48.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485591+0000", 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485592+0000", 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485598+0000", 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1070000000000001e-06 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485605+0000", 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.075e-06 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485611+0000", 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5130000000000004e-06 2024-09-18T00:57:48.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7772 3.1c 3.ec79c65c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506471+0000", 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.9884000000000001e-05, 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031227799999999998, 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7772 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506471+0000", 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506471+0000", 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506472+0000", 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5700000000000002e-07 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506477+0000", 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6760000000000004e-06 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506485+0000", 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3370000000000002e-06 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506490+0000", 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8579999999999996e-06 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7237 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484166+0000", 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022334677000000001, 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022678455, 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7237 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484166+0000", 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484166+0000", 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484167+0000", 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484169+0000", 2024-09-18T00:57:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484176+0000", 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.669e-06 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484182+0000", 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0650000000000004e-06 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7305 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484895+0000", 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021605843, 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022012167999999999, 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7305 2024-09-18T00:57:48.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484895+0000", 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484895+0000", 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484895+0000", 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0900000000000002e-07 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484898+0000", 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.362e-06 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484905+0000", 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6719999999999998e-06 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484910+0000", 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1220000000000003e-06 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7423 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020882404, 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021348407, 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7423 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:48.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485621+0000", 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485627+0000", 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.156e-06 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485633+0000", 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789999999999999e-06 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7697 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.544 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.500059+0000", 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0064422170000000001, 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0069670230000000001, 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7697 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500059+0000", 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500059+0000", 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500060+0000", 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.637e-06 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500067+0000", 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5080000000000002e-06 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500082+0000", 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4731e-05 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500091+0000", 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8279999999999998e-06 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.547 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7786 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506498+0000", 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.9179999999999998e-06, 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000603522, 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.548 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7786 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506498+0000", 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506498+0000", 2024-09-18T00:57:48.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506498+0000", 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8799999999999998e-07 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506501+0000", 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.323e-06 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506507+0000", 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0159999999999997e-06 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506512+0000", 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2410000000000003e-06 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7239 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.551 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484189+0000", 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022311620000000001, 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022998336000000001, 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7239 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484189+0000", 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484189+0000", 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484190+0000", 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-18T00:57:48.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484192+0000", 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3140000000000002e-06 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484199+0000", 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5270000000000004e-06 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484204+0000", 2024-09-18T00:57:48.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0950000000000003e-06 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7307 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484916+0000", 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021585070000000001, 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022337078999999999, 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7307 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484916+0000", 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484916+0000", 2024-09-18T00:57:48.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484916+0000", 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484918+0000", 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8929999999999999e-06 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484925+0000", 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4570000000000002e-06 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484930+0000", 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0669999999999999e-06 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.558 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7424 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020861414000000002, 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021673619000000002, 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7424 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:48.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485642+0000", 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0640000000000001e-06 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485649+0000", 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485654+0000", 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2329999999999997e-06 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7698 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.500109+0000", 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0063918059999999999, 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072630050000000003, 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7698 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500109+0000", 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500109+0000", 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500110+0000", 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6000000000000004e-07 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500113+0000", 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0110000000000001e-06 2024-09-18T00:57:48.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500121+0000", 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4360000000000002e-06 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500142+0000", 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1174000000000001e-05 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7787 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506519+0000", 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9999819, 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000909201, 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7787 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.566 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506519+0000", 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506519+0000", 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506519+0000", 2024-09-18T00:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4799999999999999e-07 2024-09-18T00:57:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506522+0000", 2024-09-18T00:57:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1490000000000001e-06 2024-09-18T00:57:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506528+0000", 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6800000000000004e-06 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506533+0000", 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1580000000000004e-06 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7255 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484210+0000", 2024-09-18T00:57:48.569 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022290556, 2024-09-18T00:57:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023275331999999999, 2024-09-18T00:57:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7255 2024-09-18T00:57:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484210+0000", 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484210+0000", 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484211+0000", 2024-09-18T00:57:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8799999999999998e-07 2024-09-18T00:57:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484213+0000", 2024-09-18T00:57:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:57:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484220+0000", 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5910000000000001e-06 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484225+0000", 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1880000000000001e-06 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7309 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484936+0000", 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021564738999999999, 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022603477, 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7309 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484936+0000", 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484936+0000", 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484937+0000", 2024-09-18T00:57:48.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2300000000000002e-07 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484939+0000", 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2029999999999999e-06 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484945+0000", 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5780000000000004e-06 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484951+0000", 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2580000000000003e-06 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020839901000000001, 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021936463999999999, 2024-09-18T00:57:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:57:48.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7699 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.500153+0000", 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0063479169999999998, 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0075105329999999998, 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7699 2024-09-18T00:57:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500153+0000", 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500153+0000", 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500154+0000", 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4300000000000003e-07 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500157+0000", 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2509999999999999e-06 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500165+0000", 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.456e-06 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500171+0000", 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0990000000000004e-06 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7788 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506540+0000", 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9999609, 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011842000000000001, 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7788 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506540+0000", 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506540+0000", 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506540+0000", 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506547+0000", 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2060000000000001e-06 2024-09-18T00:57:48.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506555+0000", 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4649999999999997e-06 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506561+0000", 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6640000000000003e-06 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7256 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484232+0000", 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022269249000000001, 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023551755000000001, 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7256 2024-09-18T00:57:48.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484232+0000", 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484232+0000", 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484232+0000", 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484234+0000", 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.119e-06 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484240+0000", 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9320000000000001e-06 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484245+0000", 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6920000000000001e-06 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7311 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484957+0000", 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021544034, 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022885398000000001, 2024-09-18T00:57:48.591 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7311 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484957+0000", 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484957+0000", 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484957+0000", 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484959+0000", 2024-09-18T00:57:48.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0849999999999999e-06 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484966+0000", 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6409999999999996e-06 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484971+0000", 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1479999999999997e-06 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020818463999999998, 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022240367, 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.595 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7700 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.500194+0000", 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0063064219999999999, 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077855320000000004, 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7700 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500194+0000", 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500194+0000", 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500195+0000", 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.51e-07 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500198+0000", 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.19e-06 2024-09-18T00:57:48.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500208+0000", 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3030000000000005e-06 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500215+0000", 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5809999999999999e-06 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7789 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506568+0000", 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9999332, 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001471626, 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7789 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506568+0000", 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506568+0000", 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506568+0000", 2024-09-18T00:57:48.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506570+0000", 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2819999999999999e-06 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506577+0000", 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5719999999999999e-06 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506582+0000", 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7257 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484252+0000", 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022249175999999999, 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023844659000000001, 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7257 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484252+0000", 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484252+0000", 2024-09-18T00:57:48.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484252+0000", 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4799999999999999e-07 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484268+0000", 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5475999999999999e-05 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484276+0000", 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.157e-06 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484282+0000", 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2140000000000001e-06 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7312 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.608 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484978+0000", 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021523141999999999, 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023175474000000001, 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7312 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484978+0000", 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484978+0000", 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484978+0000", 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.34e-07 2024-09-18T00:57:48.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484980+0000", 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2139999999999999e-06 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484987+0000", 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7059999999999998e-06 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484993+0000", 2024-09-18T00:57:48.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9950000000000002e-06 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020723136999999999, 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022417846000000002, 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7701 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.500224+0000", 2024-09-18T00:57:48.615 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.006276729, 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080064309999999996, 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7701 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500224+0000", 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500224+0000", 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500225+0000", 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500227+0000", 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.368e-06 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500234+0000", 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4529999999999999e-06 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500239+0000", 2024-09-18T00:57:48.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0529999999999997e-06 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7790 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506588+0000", 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9999123, 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016975860000000001, 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7790 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506588+0000", 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506588+0000", 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506589+0000", 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3300000000000003e-07 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506591+0000", 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2239999999999998e-06 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506598+0000", 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6839999999999999e-06 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506603+0000", 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0690000000000001e-06 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7258 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484290+0000", 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022210712000000001, 2024-09-18T00:57:48.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024027318999999998, 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7258 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484290+0000", 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484290+0000", 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484291+0000", 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5999999999999999e-07 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484293+0000", 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3819999999999998e-06 2024-09-18T00:57:48.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484299+0000", 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484304+0000", 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7949999999999998e-06 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7314 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485000+0000", 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021501379000000001, 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023350000999999999, 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7314 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.626 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485000+0000", 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485000+0000", 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485000+0000", 2024-09-18T00:57:48.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6600000000000002e-07 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485008+0000", 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7209999999999994e-06 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485015+0000", 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1620000000000002e-06 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485020+0000", 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3730000000000001e-06 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020697698, 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022592177000000001, 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.632 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7702 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.500246+0000", 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0062553069999999999, 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081941440000000004, 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7702 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500246+0000", 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500246+0000", 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500246+0000", 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.77e-07 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500248+0000", 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.046e-06 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500254+0000", 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.365e-06 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500259+0000", 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9629999999999997e-06 2024-09-18T00:57:48.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7259 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484311+0000", 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022189476999999999, 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024176850999999999, 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7259 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484311+0000", 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484311+0000", 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484312+0000", 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1300000000000001e-07 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484315+0000", 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7800000000000001e-06 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484321+0000", 2024-09-18T00:57:48.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2829999999999998e-06 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484326+0000", 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-06 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7315 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485029+0000", 2024-09-18T00:57:48.639 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021472336000000002, 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023527093999999998, 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7315 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485029+0000", 2024-09-18T00:57:48.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485029+0000", 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485029+0000", 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485031+0000", 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3369999999999998e-06 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485038+0000", 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485044+0000", 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.446e-06 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020675175, 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022773234, 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:57:48.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7703 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.500265+0000", 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.006235567, 2024-09-18T00:57:48.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083705169999999992, 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7703 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500265+0000", 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500265+0000", 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500266+0000", 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500268+0000", 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7969999999999999e-06 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500273+0000", 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500280+0000", 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4670000000000001e-06 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7260 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484332+0000", 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022169219, 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024365906999999999, 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7260 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484332+0000", 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484332+0000", 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484332+0000", 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0600000000000001e-07 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484334+0000", 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.858e-06 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484340+0000", 2024-09-18T00:57:48.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.365e-06 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484345+0000", 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7659999999999998e-06 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7321 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485050+0000", 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021450568999999999, 2024-09-18T00:57:48.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023816374000000001, 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7321 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485050+0000", 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485050+0000", 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485051+0000", 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8200000000000001e-07 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485057+0000", 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485064+0000", 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0609999999999998e-06 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485069+0000", 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020651735000000001, 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023082873, 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:57:48.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7704 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.500287+0000", 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0062141360000000003, 2024-09-18T00:57:48.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087059760000000007, 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7704 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500287+0000", 2024-09-18T00:57:48.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500287+0000", 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500287+0000", 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500290+0000", 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.278e-06 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500296+0000", 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7139999999999996e-06 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500302+0000", 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5210000000000002e-06 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.663 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7194 3.10 3:0a4d1c1d:::rbd_data.117ce245106a.000000000000000c:5 [read 589824~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.483614+0000", 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022886806999999999, 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025447872999999999, 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.664 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7194 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483614+0000", 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483614+0000", 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483615+0000", 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1100000000000001e-07 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483617+0000", 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2050000000000001e-06 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483623+0000", 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0599999999999996e-06 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483628+0000", 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8790000000000002e-06 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507369+0000", 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023741041000000001 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:48.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507582+0000", 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021353999999999999 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7261 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484352+0000", 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022148708, 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024780772, 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7261 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484352+0000", 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484352+0000", 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484353+0000", 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484355+0000", 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3860000000000001e-06 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484362+0000", 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7039999999999997e-06 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484368+0000", 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7323 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485077+0000", 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021424230999999998, 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024121460000000001, 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7323 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.672 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485077+0000", 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485077+0000", 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485077+0000", 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-07 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485079+0000", 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2500000000000001e-06 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485086+0000", 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.725e-06 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485091+0000", 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9559999999999996e-06 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020627924999999998, 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023384889999999998, 2024-09-18T00:57:48.675 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.678 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7705 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.500309+0000", 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0061922189999999997, 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090135059999999993, 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7705 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500309+0000", 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500309+0000", 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500309+0000", 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.75e-07 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500316+0000", 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500322+0000", 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7739999999999999e-06 2024-09-18T00:57:48.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.500328+0000", 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3539999999999999e-06 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7262 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484375+0000", 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022125947999999999, 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025008698999999999, 2024-09-18T00:57:48.682 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7262 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484375+0000", 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484375+0000", 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484375+0000", 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4099999999999999e-07 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484378+0000", 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.198e-06 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484384+0000", 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.827e-06 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484390+0000", 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5910000000000003e-06 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.685 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7324 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485098+0000", 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021403299000000001, 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024350424999999998, 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.686 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7324 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485098+0000", 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485098+0000", 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485098+0000", 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485100+0000", 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0470000000000001e-06 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485107+0000", 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6270000000000002e-06 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485112+0000", 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.096e-06 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020605559999999998, 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023612632000000001, 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.692 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7726 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.501631+0000", 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0048696770000000002, 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079387780000000005, 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7726 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501631+0000", 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501631+0000", 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501633+0000", 2024-09-18T00:57:48.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3650000000000001e-06 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501639+0000", 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1840000000000004e-06 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501653+0000", 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4008e-05 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501663+0000", 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7480000000000005e-06 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7819 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506734+0000", 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9997668, 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028978609999999998, 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7819 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506734+0000", 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506734+0000", 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506735+0000", 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4799999999999998e-07 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506739+0000", 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0489999999999997e-06 2024-09-18T00:57:48.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506748+0000", 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9749999999999996e-06 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506753+0000", 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6049999999999997e-06 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7263 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484397+0000", 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022103937000000001, 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025313826000000001, 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7263 2024-09-18T00:57:48.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484397+0000", 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484397+0000", 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484397+0000", 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484400+0000", 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1600000000000001e-06 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484406+0000", 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6780000000000003e-06 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484411+0000", 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.096e-06 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7326 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485119+0000", 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021382083999999999, 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024653062, 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7326 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485119+0000", 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485119+0000", 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485119+0000", 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485121+0000", 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1509999999999998e-06 2024-09-18T00:57:48.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485128+0000", 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4699999999999999e-06 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485133+0000", 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9060000000000001e-06 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020578942999999999, 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023912625999999999, 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.707 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7727 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.501680+0000", 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0048208030000000002, 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082126990000000004, 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7727 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501680+0000", 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501680+0000", 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501681+0000", 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7800000000000001e-07 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501684+0000", 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6780000000000002e-06 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501706+0000", 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1284000000000002e-05 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501714+0000", 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0660000000000004e-06 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7820 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.713 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506762+0000", 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9997387, 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031930930000000001, 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7820 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506762+0000", 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506762+0000", 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506763+0000", 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-07 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506765+0000", 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5000000000000002e-06 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506772+0000", 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9260000000000003e-06 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506777+0000", 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-06 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.716 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7264 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484418+0000", 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022082816000000002, 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025594414999999999, 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7264 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484418+0000", 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484418+0000", 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484419+0000", 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484421+0000", 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3039999999999999e-06 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484427+0000", 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5849999999999996e-06 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484433+0000", 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0839999999999999e-06 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7348 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485139+0000", 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021361517, 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024932808000000001, 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7348 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485139+0000", 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485139+0000", 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485140+0000", 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.46e-07 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485142+0000", 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1380000000000001e-06 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485148+0000", 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2439999999999998e-06 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485153+0000", 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.977e-06 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.723 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7460 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.489809+0000", 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016692103, 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020320397, 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7460 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489809+0000", 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489809+0000", 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489811+0000", 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.61e-06 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489820+0000", 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5310000000000001e-06 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489828+0000", 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3049999999999993e-06 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489834+0000", 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.976e-06 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.726 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7728 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.501725+0000", 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0047754970000000001, 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084623860000000006, 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7728 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501725+0000", 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501725+0000", 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501726+0000", 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7999999999999995e-07 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501729+0000", 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2569999999999999e-06 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501738+0000", 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6030000000000005e-06 2024-09-18T00:57:48.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501745+0000", 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6780000000000003e-06 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7821 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506785+0000", 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9997163, 2024-09-18T00:57:48.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034635540000000002, 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7821 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506785+0000", 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506785+0000", 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506785+0000", 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506788+0000", 2024-09-18T00:57:48.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506794+0000", 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.72e-06 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506800+0000", 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2730000000000002e-06 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7265 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484439+0000", 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022061951999999999, 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025868852000000001, 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7265 2024-09-18T00:57:48.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484439+0000", 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484439+0000", 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484439+0000", 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9299999999999999e-07 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484441+0000", 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0849999999999999e-06 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484448+0000", 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.613e-06 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484453+0000", 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2299999999999999e-06 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7349 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485160+0000", 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021340681, 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025195565999999999, 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7349 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485160+0000", 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485160+0000", 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485161+0000", 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0499999999999999e-07 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485163+0000", 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2520000000000002e-06 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485170+0000", 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8959999999999997e-06 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485176+0000", 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1469999999999998e-06 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7461 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.740 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.489845+0000", 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016656250000000001, 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020552975000000001, 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7461 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489845+0000", 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489845+0000", 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489845+0000", 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489848+0000", 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0000000000000001e-06 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489856+0000", 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6769999999999995e-06 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489862+0000", 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1419999999999998e-06 2024-09-18T00:57:48.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7729 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.501763+0000", 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0047374649999999997, 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086684929999999993, 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7729 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501763+0000", 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501763+0000", 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501764+0000", 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0699999999999997e-07 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501767+0000", 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1030000000000002e-06 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501776+0000", 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0790000000000007e-06 2024-09-18T00:57:48.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501784+0000", 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4379999999999998e-06 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7822 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506835+0000", 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9996662, 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0036507620000000001, 2024-09-18T00:57:48.747 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7822 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506835+0000", 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506835+0000", 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506836+0000", 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506842+0000", 2024-09-18T00:57:48.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506864+0000", 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1348e-05 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506877+0000", 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2704000000000001e-05 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7266 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484460+0000", 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022040785, 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026086066000000002, 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7266 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.751 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484460+0000", 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484460+0000", 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484461+0000", 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484463+0000", 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4320000000000002e-06 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484470+0000", 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5610000000000004e-06 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484475+0000", 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0860000000000001e-06 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7350 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485183+0000", 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021317564000000001, 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025418809000000001, 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7350 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485183+0000", 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485183+0000", 2024-09-18T00:57:48.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485184+0000", 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7599999999999998e-07 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485186+0000", 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1119999999999999e-06 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485192+0000", 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5200000000000003e-06 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485198+0000", 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5520000000000004e-06 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7462 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.489871+0000", 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016629626000000002, 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020810783999999999, 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7462 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489871+0000", 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489871+0000", 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489872+0000", 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.75e-07 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489874+0000", 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.424e-06 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489881+0000", 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1450000000000002e-06 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489887+0000", 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9320000000000001e-06 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7730 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.501793+0000", 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0047077009999999999, 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089495129999999992, 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7730 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501793+0000", 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501793+0000", 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501794+0000", 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2499999999999995e-07 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501796+0000", 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6400000000000001e-06 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501804+0000", 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.841e-06 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501817+0000", 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2850000000000001e-05 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7267 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484482+0000", 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022018774000000001, 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026316961, 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7267 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484482+0000", 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484482+0000", 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484483+0000", 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484486+0000", 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1420000000000002e-06 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484493+0000", 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8020000000000003e-06 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484497+0000", 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8829999999999997e-06 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7351 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485205+0000", 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021296254000000001, 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025653608000000001, 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7351 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485205+0000", 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485205+0000", 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485205+0000", 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.32e-07 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485207+0000", 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1560000000000002e-06 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485214+0000", 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7530000000000004e-06 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485219+0000", 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3109999999999996e-06 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7463 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.489895+0000", 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016605587000000002, 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021030637000000001, 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7463 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489895+0000", 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489895+0000", 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489896+0000", 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1900000000000003e-07 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489899+0000", 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.824e-06 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489906+0000", 2024-09-18T00:57:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2350000000000002e-06 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489912+0000", 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7769999999999999e-06 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7731 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.501827+0000", 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0046742509999999999, 2024-09-18T00:57:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091535120000000008, 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7731 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501827+0000", 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501827+0000", 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501827+0000", 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6199999999999998e-07 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501830+0000", 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9780000000000001e-06 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501847+0000", 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7147e-05 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501858+0000", 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.101e-05 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7268 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484504+0000", 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021996562000000001, 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026541889999999999, 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7268 2024-09-18T00:57:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484504+0000", 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484504+0000", 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484505+0000", 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8000000000000001e-07 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484507+0000", 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.52e-06 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484514+0000", 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6320000000000002e-06 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484519+0000", 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1340000000000003e-06 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7352 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485226+0000", 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021275392000000001, 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025883746999999999, 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7352 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485226+0000", 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485226+0000", 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485226+0000", 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6100000000000002e-07 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485228+0000", 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0370000000000001e-06 2024-09-18T00:57:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485235+0000", 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6640000000000001e-06 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485240+0000", 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2519999999999999e-06 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7464 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.489920+0000", 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016581396000000002, 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021250897000000001, 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7464 2024-09-18T00:57:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489920+0000", 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489920+0000", 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489920+0000", 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5499999999999998e-07 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489923+0000", 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7020000000000001e-06 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489930+0000", 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2490000000000004e-06 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489935+0000", 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4779999999999999e-06 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7732 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.501869+0000", 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0046320739999999999, 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0093477419999999992, 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7732 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501869+0000", 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501869+0000", 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501869+0000", 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4300000000000003e-07 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501873+0000", 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2830000000000002e-06 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501881+0000", 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501888+0000", 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5220000000000004e-06 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7270 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484526+0000", 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021974504999999998, 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026750178999999999, 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7270 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484526+0000", 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484526+0000", 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484527+0000", 2024-09-18T00:57:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484538+0000", 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1209999999999999e-05 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484546+0000", 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6699999999999994e-06 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484552+0000", 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8590000000000001e-06 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7365 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485247+0000", 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021254064, 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026088516999999999, 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7365 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485247+0000", 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485247+0000", 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485247+0000", 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0200000000000003e-07 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485254+0000", 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9600000000000003e-06 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485262+0000", 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4889999999999998e-06 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485267+0000", 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6180000000000003e-06 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7466 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.489946+0000", 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016555018000000001, 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021448634000000001, 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7466 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489946+0000", 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489946+0000", 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489946+0000", 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1399999999999997e-07 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489949+0000", 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6280000000000001e-06 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489956+0000", 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.294e-06 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489963+0000", 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-18T00:57:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7733 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.501897+0000", 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0046038140000000003, 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0095565730000000005, 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7733 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501897+0000", 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501897+0000", 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501898+0000", 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8299999999999997e-07 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501900+0000", 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5940000000000001e-06 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501908+0000", 2024-09-18T00:57:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9829999999999996e-06 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501915+0000", 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9450000000000004e-06 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7271 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484560+0000", 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021941061000000001, 2024-09-18T00:57:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026952953000000002, 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7271 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484560+0000", 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484560+0000", 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484560+0000", 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0200000000000002e-07 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484563+0000", 2024-09-18T00:57:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.367e-06 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484569+0000", 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7689999999999999e-06 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484575+0000", 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1240000000000004e-06 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7366 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485276+0000", 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021225267999999999, 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026296127999999998, 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7366 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485276+0000", 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485276+0000", 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485276+0000", 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3700000000000001e-07 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485278+0000", 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1770000000000001e-06 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485285+0000", 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4119999999999998e-06 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485290+0000", 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2750000000000003e-06 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7489 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.489970+0000", 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016531210000000001, 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021661585000000001, 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7489 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489970+0000", 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489970+0000", 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489970+0000", 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489973+0000", 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0769999999999999e-06 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489980+0000", 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.37e-06 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489984+0000", 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9010000000000002e-06 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7734 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.501929+0000", 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0045722419999999998, 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0097621119999999999, 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7734 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501929+0000", 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501929+0000", 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501929+0000", 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6899999999999997e-07 2024-09-18T00:57:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501932+0000", 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9409999999999999e-06 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501941+0000", 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5169999999999999e-06 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.501949+0000", 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7749999999999993e-06 2024-09-18T00:57:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7272 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484582+0000", 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021919259999999999, 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027169237999999998, 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7272 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484582+0000", 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484582+0000", 2024-09-18T00:57:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484582+0000", 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484584+0000", 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9860000000000001e-06 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484591+0000", 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7379999999999997e-06 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484600+0000", 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1050000000000001e-06 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7367 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485297+0000", 2024-09-18T00:57:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021203954000000001, 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026512774999999999, 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7367 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485297+0000", 2024-09-18T00:57:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485297+0000", 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485297+0000", 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.08e-07 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485300+0000", 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1919999999999999e-06 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485306+0000", 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8569999999999997e-06 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485312+0000", 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4999999999999999e-06 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7490 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.489991+0000", 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.01650956, 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021874635, 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7490 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489991+0000", 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489991+0000", 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489992+0000", 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7100000000000002e-07 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489994+0000", 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8470000000000001e-06 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490000+0000", 2024-09-18T00:57:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.993e-06 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490005+0000", 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.853e-06 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7745 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.503848+0000", 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.002653107, 2024-09-18T00:57:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080712030000000008, 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7745 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.503848+0000", 2024-09-18T00:57:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.503848+0000", 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.503849+0000", 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.503854+0000", 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1479999999999997e-06 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.503867+0000", 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3128e-05 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.503875+0000", 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3469999999999999e-06 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7275 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484608+0000", 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021892637, 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027369586000000001, 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7275 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484608+0000", 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484608+0000", 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484609+0000", 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2800000000000002e-07 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484611+0000", 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6000000000000001e-06 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484619+0000", 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2559999999999996e-06 2024-09-18T00:57:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484624+0000", 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0830000000000003e-06 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7368 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485319+0000", 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021182308, 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026717925, 2024-09-18T00:57:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7368 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485319+0000", 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485319+0000", 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485319+0000", 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485325+0000", 2024-09-18T00:57:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6509999999999998e-06 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485332+0000", 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1400000000000002e-06 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485337+0000", 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.101e-06 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7495 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490011+0000", 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016489676000000002, 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022084672, 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7495 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490011+0000", 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490011+0000", 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490012+0000", 2024-09-18T00:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8599999999999999e-07 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490014+0000", 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.187e-06 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490020+0000", 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1240000000000002e-06 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490025+0000", 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1320000000000002e-06 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7746 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.503907+0000", 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0025935239999999998, 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082477939999999993, 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7746 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.503907+0000", 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.503907+0000", 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.503908+0000", 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6499999999999998e-07 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.503914+0000", 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.383e-06 2024-09-18T00:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504042+0000", 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012859500000000001 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504050+0000", 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3550000000000005e-06 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7285 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484631+0000", 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021870165, 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027586511000000001, 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7285 2024-09-18T00:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484631+0000", 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484631+0000", 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484631+0000", 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6699999999999999e-07 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484633+0000", 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1320000000000001e-06 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484640+0000", 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7190000000000004e-06 2024-09-18T00:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484646+0000", 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4779999999999999e-06 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7383 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485345+0000", 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021156041, 2024-09-18T00:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026930778999999998, 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7383 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485345+0000", 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485345+0000", 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485345+0000", 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485348+0000", 2024-09-18T00:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3580000000000001e-06 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485355+0000", 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2300000000000002e-06 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485360+0000", 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3489999999999999e-06 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7496 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490033+0000", 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016468231, 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022303008999999999, 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7496 2024-09-18T00:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490033+0000", 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490033+0000", 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490033+0000", 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6100000000000001e-07 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490035+0000", 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490042+0000", 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490046+0000", 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5000000000000001e-06 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7747 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.504097+0000", 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0024042239999999999, 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082977959999999996, 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7747 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504097+0000", 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504097+0000", 2024-09-18T00:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504097+0000", 2024-09-18T00:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9400000000000005e-07 2024-09-18T00:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504101+0000", 2024-09-18T00:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0369999999999996e-06 2024-09-18T00:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504112+0000", 2024-09-18T00:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0560999999999999e-05 2024-09-18T00:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504119+0000", 2024-09-18T00:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-18T00:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7216 3.10 3:0a4d1c1d:::rbd_data.117ce245106a.000000000000000c:5 [read 1081344~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.483891+0000", 2024-09-18T00:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022610392999999999, 2024-09-18T00:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028569635999999999, 2024-09-18T00:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7216 2024-09-18T00:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483891+0000", 2024-09-18T00:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483891+0000", 2024-09-18T00:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483891+0000", 2024-09-18T00:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8999999999999997e-07 2024-09-18T00:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483893+0000", 2024-09-18T00:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483900+0000", 2024-09-18T00:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8650000000000003e-06 2024-09-18T00:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483905+0000", 2024-09-18T00:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2320000000000001e-06 2024-09-18T00:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.511783+0000", 2024-09-18T00:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027877967999999999 2024-09-18T00:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7288 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484653+0000", 2024-09-18T00:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021848223999999999, 2024-09-18T00:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027874717, 2024-09-18T00:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7288 2024-09-18T00:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484653+0000", 2024-09-18T00:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484653+0000", 2024-09-18T00:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484653+0000", 2024-09-18T00:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5200000000000002e-07 2024-09-18T00:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484655+0000", 2024-09-18T00:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1280000000000002e-06 2024-09-18T00:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484662+0000", 2024-09-18T00:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4679999999999997e-06 2024-09-18T00:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484667+0000", 2024-09-18T00:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8670000000000002e-06 2024-09-18T00:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7384 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485367+0000", 2024-09-18T00:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021133543000000001, 2024-09-18T00:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027218022000000001, 2024-09-18T00:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7384 2024-09-18T00:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485367+0000", 2024-09-18T00:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485367+0000", 2024-09-18T00:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485368+0000", 2024-09-18T00:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5200000000000002e-07 2024-09-18T00:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485370+0000", 2024-09-18T00:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9549999999999999e-06 2024-09-18T00:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485376+0000", 2024-09-18T00:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0979999999999999e-06 2024-09-18T00:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485381+0000", 2024-09-18T00:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2070000000000003e-06 2024-09-18T00:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7497 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490052+0000", 2024-09-18T00:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016448745000000001, 2024-09-18T00:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022588075999999999, 2024-09-18T00:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7497 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490052+0000", 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490052+0000", 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490053+0000", 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490055+0000", 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4870000000000001e-06 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490062+0000", 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8299999999999998e-06 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490067+0000", 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1080000000000001e-06 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7748 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.504130+0000", 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.002370549, 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085758959999999995, 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7748 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504130+0000", 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504130+0000", 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504131+0000", 2024-09-18T00:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6400000000000002e-07 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504134+0000", 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3670000000000001e-06 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504142+0000", 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504155+0000", 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3614999999999999e-05 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7217 3.10 3:0a4d1c1d:::rbd_data.117ce245106a.000000000000000c:5 [read 1146880~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.483912+0000", 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022588909000000001, 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028865089999999999, 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7217 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483912+0000", 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483912+0000", 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483912+0000", 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7399999999999999e-07 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483914+0000", 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9230000000000001e-06 2024-09-18T00:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483921+0000", 2024-09-18T00:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2269999999999998e-06 2024-09-18T00:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483926+0000", 2024-09-18T00:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6420000000000004e-06 2024-09-18T00:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.512722+0000", 2024-09-18T00:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028795584999999999 2024-09-18T00:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7289 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484674+0000", 2024-09-18T00:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021827210999999999, 2024-09-18T00:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028171222999999999, 2024-09-18T00:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7289 2024-09-18T00:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484674+0000", 2024-09-18T00:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484674+0000", 2024-09-18T00:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484674+0000", 2024-09-18T00:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T00:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484676+0000", 2024-09-18T00:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.176e-06 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484683+0000", 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4250000000000003e-06 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484688+0000", 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9899999999999997e-06 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7385 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485388+0000", 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021112493999999999, 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027521833999999998, 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7385 2024-09-18T00:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485388+0000", 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485388+0000", 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485389+0000", 2024-09-18T00:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-18T00:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485391+0000", 2024-09-18T00:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2110000000000001e-06 2024-09-18T00:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485398+0000", 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6819999999999997e-06 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485403+0000", 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2460000000000003e-06 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016427172, 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022885540999999999, 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7749 3.10 3.80950e90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.504164+0000", 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0023370219999999998, 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088284139999999997, 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7749 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504164+0000", 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504164+0000", 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504164+0000", 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504167+0000", 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4949999999999998e-06 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504174+0000", 2024-09-18T00:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7279999999999998e-06 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.504178+0000", 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9250000000000003e-06 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7226 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.483933+0000", 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022567729000000002, 2024-09-18T00:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029101214, 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7226 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483933+0000", 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483933+0000", 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483934+0000", 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483937+0000", 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2480000000000001e-06 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483944+0000", 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6529999999999997e-06 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483949+0000", 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7320000000000003e-06 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7290 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484695+0000", 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021805886, 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028370245999999998, 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7290 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484695+0000", 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484695+0000", 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484695+0000", 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2500000000000001e-07 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484698+0000", 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4040000000000002e-06 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484705+0000", 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7789999999999998e-06 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484710+0000", 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3240000000000002e-06 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7386 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485410+0000", 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021091155, 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027718418000000002, 2024-09-18T00:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7386 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485410+0000", 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485410+0000", 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485410+0000", 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7500000000000001e-07 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485412+0000", 2024-09-18T00:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.874e-06 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485418+0000", 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0680000000000002e-06 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485423+0000", 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1749999999999996e-06 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016406429, 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02309369, 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7750 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506170+0000", 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.00033075599999999998, 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070806469999999998, 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7750 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506170+0000", 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506170+0000", 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506171+0000", 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1319999999999999e-06 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506180+0000", 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7390000000000005e-06 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506194+0000", 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3677e-05 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506202+0000", 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6060000000000003e-06 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7864 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.507379+0000", 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9991221, 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0059325309999999996, 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7864 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507379+0000", 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507379+0000", 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507380+0000", 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5300000000000004e-07 2024-09-18T00:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507385+0000", 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9050000000000002e-06 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507395+0000", 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9019999999999994e-06 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507403+0000", 2024-09-18T00:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5549999999999997e-06 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7227 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.483957+0000", 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022544066000000001, 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029412619000000001, 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7227 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483957+0000", 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483957+0000", 2024-09-18T00:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483957+0000", 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3300000000000003e-07 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483974+0000", 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7167e-05 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483983+0000", 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.619e-06 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483989+0000", 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7291 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484717+0000", 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021783622999999998, 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028709944000000001, 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7291 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484717+0000", 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484717+0000", 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484718+0000", 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484720+0000", 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0959999999999999e-06 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484726+0000", 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.19e-06 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484731+0000", 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9660000000000004e-06 2024-09-18T00:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7402 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485430+0000", 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021070569000000001, 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028052726, 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7402 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485430+0000", 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485430+0000", 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485431+0000", 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1300000000000001e-07 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485433+0000", 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.429e-06 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485440+0000", 2024-09-18T00:57:48.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8020000000000003e-06 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485445+0000", 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4960000000000004e-06 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016360863999999999, 2024-09-18T00:57:48.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023397134999999999, 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:48.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7751 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506220+0000", 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.00028125999999999997, 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0073763359999999998, 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7751 2024-09-18T00:57:48.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506220+0000", 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506220+0000", 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506220+0000", 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8599999999999998e-07 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506223+0000", 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.163e-06 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506232+0000", 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8969999999999997e-06 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506239+0000", 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3210000000000001e-06 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7865 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.507413+0000", 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9990873, 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062404280000000001, 2024-09-18T00:57:48.944 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7865 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507413+0000", 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507413+0000", 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507414+0000", 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9599999999999999e-07 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507417+0000", 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9840000000000001e-06 2024-09-18T00:57:48.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507427+0000", 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5820000000000008e-06 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507434+0000", 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4050000000000003e-06 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.947 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7228 3.10 3:0a4d1c1d:::rbd_data.117ce245106a.000000000000000c:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.483998+0000", 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.02250245, 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029734340000000001, 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7228 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483998+0000", 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483998+0000", 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.483999+0000", 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1500000000000005e-07 2024-09-18T00:57:48.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484002+0000", 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1420000000000002e-06 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484009+0000", 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7139999999999996e-06 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484014+0000", 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4940000000000003e-06 2024-09-18T00:57:48.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.513659+0000", 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029644750000000001 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7294 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484738+0000", 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021763257000000001, 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029034051000000002, 2024-09-18T00:57:48.951 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7294 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484738+0000", 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484738+0000", 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484738+0000", 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3700000000000001e-07 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484740+0000", 2024-09-18T00:57:48.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1100000000000001e-06 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484747+0000", 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4590000000000003e-06 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484752+0000", 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0819999999999998e-06 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7403 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485453+0000", 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021048404, 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028349556000000001, 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7403 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485453+0000", 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485453+0000", 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485453+0000", 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8599999999999999e-07 2024-09-18T00:57:48.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485455+0000", 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9460000000000001e-06 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485461+0000", 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4899999999999997e-06 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485467+0000", 2024-09-18T00:57:48.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016334523, 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023665418000000001, 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:57:48.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7752 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.961 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506247+0000", 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.00025371699999999998, 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0076141830000000001, 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7752 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506247+0000", 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506247+0000", 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506248+0000", 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.08e-07 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506250+0000", 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4329999999999998e-06 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506257+0000", 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8950000000000001e-06 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506262+0000", 2024-09-18T00:57:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0590000000000002e-06 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7866 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.507446+0000", 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9990544, 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0064477379999999997, 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7866 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507446+0000", 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507446+0000", 2024-09-18T00:57:48.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507447+0000", 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8100000000000003e-07 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507451+0000", 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2030000000000002e-06 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507462+0000", 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0971999999999999e-05 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507471+0000", 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5529999999999993e-06 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7229 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484023+0000", 2024-09-18T00:57:48.968 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022477941000000001, 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029931368, 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7229 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484023+0000", 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484023+0000", 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484023+0000", 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484035+0000", 2024-09-18T00:57:48.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2062000000000001e-05 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484043+0000", 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5560000000000002e-06 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484049+0000", 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6740000000000002e-06 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.971 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7296 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484759+0000", 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021742378999999999, 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029257729999999999, 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7296 2024-09-18T00:57:48.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484759+0000", 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484759+0000", 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484759+0000", 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0999999999999999e-07 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484762+0000", 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.134e-06 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484768+0000", 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3459999999999999e-06 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484774+0000", 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8080000000000001e-06 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7404 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485473+0000", 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021027575999999999, 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028607127, 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7404 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485473+0000", 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485473+0000", 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485474+0000", 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0900000000000002e-07 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485476+0000", 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0889999999999998e-06 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485483+0000", 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8079999999999999e-06 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485488+0000", 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.366e-06 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016311853000000001, 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023951146999999999, 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:57:48.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:57:48.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7753 3.10 3.b838b250 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.506269+0000", 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.000232296, 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079291640000000007, 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7753 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506269+0000", 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506269+0000", 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506269+0000", 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506286+0000", 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6614999999999999e-05 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506294+0000", 2024-09-18T00:57:48.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7199999999999995e-06 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.506299+0000", 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1449999999999999e-06 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7867 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.507485+0000", 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9990163, 2024-09-18T00:57:48.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0067619100000000003, 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7867 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507485+0000", 2024-09-18T00:57:48.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507485+0000", 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507485+0000", 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507515+0000", 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9581999999999999e-05 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507528+0000", 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2904e-05 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507537+0000", 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6179999999999995e-06 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7230 3.10 3:0a4d1c1d:::rbd_data.117ce245106a.000000000000000c:5 [read 196608~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484057+0000", 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022443634000000001, 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030240522999999998, 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7230 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484057+0000", 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484057+0000", 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484058+0000", 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8599999999999998e-07 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484060+0000", 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5059999999999998e-06 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484067+0000", 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7549999999999997e-06 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484072+0000", 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3040000000000004e-06 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.514256+0000", 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030183742999999999 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.992 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7297 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484781+0000", 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021719538, 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029555306999999999, 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7297 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484781+0000", 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484781+0000", 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484782+0000", 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6199999999999998e-07 2024-09-18T00:57:48.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484784+0000", 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1090000000000001e-06 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484790+0000", 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2620000000000004e-06 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484796+0000", 2024-09-18T00:57:48.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6060000000000002e-06 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7405 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485494+0000", 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021006720999999999, 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028873586999999999, 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:48.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7405 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485494+0000", 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485494+0000", 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:48.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485495+0000", 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6300000000000001e-07 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485497+0000", 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6759999999999999e-06 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485504+0000", 2024-09-18T00:57:48.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6900000000000003e-06 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485509+0000", 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3709999999999999e-06 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016293479999999999, 2024-09-18T00:57:48.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024342320000000001, 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:49.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7868 3.10 3.fbc2c550 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.507550+0000", 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4294967295.9989514, 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070658170000000003, 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7868 2024-09-18T00:57:49.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507550+0000", 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507550+0000", 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507550+0000", 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8999999999999995e-07 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507554+0000", 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3289999999999998e-06 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507563+0000", 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7249999999999992e-06 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.507571+0000", 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1760000000000004e-06 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7231 3.10 3:0970a901:::rbd_data.117ce245106a.0000000000000009:5 [read 1146880~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484080+0000", 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022420450000000001, 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030603018999999999, 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7231 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484080+0000", 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484080+0000", 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484081+0000", 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484083+0000", 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3549999999999999e-06 2024-09-18T00:57:49.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484090+0000", 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7989999999999996e-06 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484095+0000", 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8729999999999998e-06 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.514572+0000", 2024-09-18T00:57:49.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030476627999999999 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.514655+0000", 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3206000000000003e-05 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7299 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484808+0000", 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021693222000000002, 2024-09-18T00:57:49.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029960962000000001, 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7299 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484808+0000", 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484808+0000", 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484808+0000", 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3599999999999999e-07 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484811+0000", 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.655e-06 2024-09-18T00:57:49.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484818+0000", 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1690000000000003e-06 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484823+0000", 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1039999999999998e-06 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7406 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485517+0000", 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020983785000000001, 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029315371, 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7406 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.014 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485517+0000", 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485517+0000", 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485518+0000", 2024-09-18T00:57:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6499999999999999e-07 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485525+0000", 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0820000000000001e-06 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485532+0000", 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7640000000000005e-06 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485538+0000", 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7769999999999999e-06 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016273691999999999, 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024665474999999999, 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:49.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7300 3.10 3.b838b250 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484831+0000", 2024-09-18T00:57:49.020 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021670137999999999, 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030114640000000002, 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7300 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484831+0000", 2024-09-18T00:57:49.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484831+0000", 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484831+0000", 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3700000000000001e-07 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484833+0000", 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.187e-06 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484840+0000", 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484845+0000", 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1390000000000003e-06 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7420 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485546+0000", 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020954555, 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029475843000000002, 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7420 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.024 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485546+0000", 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485546+0000", 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485547+0000", 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485549+0000", 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1909999999999999e-06 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485556+0000", 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6309999999999997e-06 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485561+0000", 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4310000000000002e-06 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.016246822000000001, 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024832316, 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:49.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.030 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7234 3.10 3:0a4d1c1d:::rbd_data.117ce245106a.000000000000000c:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484123+0000", 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.022377993999999998, 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.031015533000000001, 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7234 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484123+0000", 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484123+0000", 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484123+0000", 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0600000000000001e-07 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484126+0000", 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.446e-06 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484133+0000", 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9829999999999999e-06 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484138+0000", 2024-09-18T00:57:49.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2680000000000002e-06 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.514941+0000", 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030802658 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.515034+0000", 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3289999999999996e-05 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7302 3.10 3.80950e90 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.484853+0000", 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.021648190000000001, 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030377357000000001, 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7302 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.035 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484853+0000", 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484853+0000", 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484853+0000", 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:57:49.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484855+0000", 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2670000000000001e-06 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484862+0000", 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2639999999999997e-06 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.484867+0000", 2024-09-18T00:57:49.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0309999999999998e-06 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7421 3.10 3.d2b5cfb0 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485568+0000", 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020932796, 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029733896999999999, 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "queued for pg", 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7421 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485568+0000", 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485568+0000", 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485569+0000", 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-07 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485571+0000", 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3549999999999999e-06 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485578+0000", 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2350000000000002e-06 2024-09-18T00:57:49.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485584+0000", 2024-09-18T00:57:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0990000000000004e-06 2024-09-18T00:57:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:57:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 147 2024-09-18T00:57:49.041 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:49.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:57:49.044 INFO:teuthology.orchestra.run.smithi042.stdout:271 writesame 0x2c4fe9 thru 0x2c84b6 data_size 0x3(0x34ce bytes) 2024-09-18T00:57:49.044 INFO:teuthology.orchestra.run.smithi042.stdout:272 punch from 0x4d8c275 to 0x4d913db, (0x5166 bytes) 2024-09-18T00:57:49.044 INFO:teuthology.orchestra.run.smithi042.stdout:274 write 0xe574a87 thru 0xe57e45f (0x99d9 bytes) 2024-09-18T00:57:49.044 INFO:teuthology.orchestra.run.smithi042.stdout:275 write 0xb1faf01 thru 0xb2082ac (0xd3ac bytes) 2024-09-18T00:57:49.044 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:49.038+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:49.144 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:49.138+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:49.215 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:49.215 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:49.215 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:49.215 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:49.215 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:49.222 INFO:teuthology.orchestra.run.smithi042.stdout:279 trunc from 0xe57e460 to 0x4659058 2024-09-18T00:57:49.235 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:57:49.282 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:49.278+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:49.315 INFO:teuthology.orchestra.run.smithi042.stdout:280 punch from 0x37888f to 0x3887c7, (0xff38 bytes) 2024-09-18T00:57:49.331 INFO:teuthology.orchestra.run.smithi042.stdout:281 writesame 0xa87fb30 thru 0xa88b113 data_size 0xe(0xb5e4 bytes) 2024-09-18T00:57:49.343 INFO:teuthology.orchestra.run.smithi042.stdout:283 write 0xd02bbfa thru 0xd034bfa (0x9001 bytes) 2024-09-18T00:57:49.345 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:49.342+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:49.348 INFO:teuthology.orchestra.run.smithi042.stdout:284 flatten 2024-09-18T00:57:49.395 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:49.396 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:57:49.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:57:49.396 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:49.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.396 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:49.396 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:49.396 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.103800399000001, 2024-09-18T00:57:49.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:57:49.396 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:49.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:57:49.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:49.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.402 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7423 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90750184899999997, 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045296661000000002, 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7423 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-18T00:57:49.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485621+0000", 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485627+0000", 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.156e-06 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485633+0000", 2024-09-18T00:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789999999999999e-06 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530788+0000", 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045155001 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530853+0000", 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5634000000000002e-05 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530915+0000", 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2022999999999999e-05 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7424 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1146880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90748085899999997, 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045419355000000002, 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.407 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7424 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485642+0000", 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0640000000000001e-06 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485649+0000", 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485654+0000", 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2329999999999997e-06 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530941+0000", 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045286602000000002 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531004+0000", 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3455999999999997e-05 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531059+0000", 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4771000000000001e-05 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.411 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.907459346, 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:57:49.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:57:49.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90743790899999999, 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:57:49.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90734258199999995, 2024-09-18T00:57:49.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:57:49.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.424 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90731714299999999, 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.425 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:57:49.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90729462000000005, 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:57:49.429 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:57:49.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.433 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90727117999999995, 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:57:49.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:49.438 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90724737, 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:49.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:57:49.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90722500500000003, 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90719838799999997, 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:57:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:57:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.903046617, 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:57:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90302587400000001, 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:57:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:57:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:57:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:57:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90298030900000004, 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:57:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90295396800000005, 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:57:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90293129800000005, 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:57:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90291292499999998, 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90289313699999996, 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90286626699999994, 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:57:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:57:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:57:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:49.488 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:49.490 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:57:49.490 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:49.486+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:49.496 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:57:49.590 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:49.586+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:49.605 INFO:teuthology.orchestra.run.smithi042.stdout:285 write 0x7d46d01 thru 0x7d52ea7 (0xc1a7 bytes) 2024-09-18T00:57:49.610 INFO:teuthology.orchestra.run.smithi042.stdout:286 punch from 0x16709f9 to 0x16787ba, (0x7dc1 bytes) 2024-09-18T00:57:49.610 INFO:teuthology.orchestra.run.smithi042.stdout:288 punch from 0x3cf4ef2 to 0x3cf6420, (0x152e bytes) 2024-09-18T00:57:49.610 INFO:teuthology.orchestra.run.smithi042.stdout:289 read 0x415f1af thru 0x416dbcd (0xea1f bytes) 2024-09-18T00:57:49.614 INFO:teuthology.orchestra.run.smithi042.stdout:290 trunc from 0xd034bfb to 0xe8e17cd 2024-09-18T00:57:49.624 INFO:teuthology.orchestra.run.smithi042.stdout:291 write 0xa8f7848 thru 0xa901a78 (0xa231 bytes) 2024-09-18T00:57:49.632 INFO:teuthology.orchestra.run.smithi042.stdout:292 punch from 0xbeadac5 to 0xbeafc65, (0x21a0 bytes) 2024-09-18T00:57:49.632 INFO:teuthology.orchestra.run.smithi042.stdout:293 write 0x2b5687b thru 0x2b58be3 (0x2369 bytes) 2024-09-18T00:57:49.635 INFO:teuthology.orchestra.run.smithi042.stdout:294 punch from 0x2cda437 to 0x2ce4bb4, (0xa77d bytes) 2024-09-18T00:57:49.636 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:49.636 INFO:teuthology.orchestra.run.smithi042.stdout:296 read 0x195b0ee thru 0x1968545 (0xd458 bytes) 2024-09-18T00:57:49.637 INFO:teuthology.orchestra.run.smithi042.stdout:298 writesame 0x3e51618 thru 0x3e59050 data_size 0x1(0x7a39 bytes) 2024-09-18T00:57:49.668 INFO:teuthology.orchestra.run.smithi042.stdout:299 read 0x8f585e thru 0x8f5cb8 (0x45b bytes) 2024-09-18T00:57:49.669 INFO:teuthology.orchestra.run.smithi042.stdout:300 writesame 0x8aa8bc5 thru 0x8ab8958 data_size 0x2(0xfd94 bytes) 2024-09-18T00:57:49.674 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:49.674 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:49.674 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:49.674 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:49.689 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:57:49.689 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:57:49.691 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:49.686+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:49.696 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:57:49.703 INFO:teuthology.orchestra.run.smithi042.stdout:301 trunc from 0xe8e17cd to 0x304cb44 2024-09-18T00:57:49.711 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:57:49.711 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:57:49.729 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:49.730 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:57:49.730 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:57:49.738 INFO:teuthology.orchestra.run.smithi042.stdout:303 write 0x589fa29 thru 0x58a8c08 (0x91e0 bytes) 2024-09-18T00:57:49.745 INFO:teuthology.orchestra.run.smithi042.stdout:304 trunc from 0x58a8c09 to 0x770978d 2024-09-18T00:57:49.750 INFO:teuthology.orchestra.run.smithi042.stdout:305 read 0x4efa63c thru 0x4f00249 (0x5c0e bytes) 2024-09-18T00:57:49.751 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:57:49.751 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:57:49.751 INFO:teuthology.orchestra.run.smithi042.stdout:307 read 0x141b1dd thru 0x141e928 (0x374c bytes) 2024-09-18T00:57:49.752 INFO:teuthology.orchestra.run.smithi042.stdout:308 writesame 0x1f9856f thru 0x1fa19a1 data_size 0x1(0x9433 bytes) 2024-09-18T00:57:49.773 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:57:49.773 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:57:49.792 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:49.786+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:49.798 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:49.803 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:57:49.803 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:57:49.805 INFO:teuthology.orchestra.run.smithi042.stdout:314 write 0x813e534 thru 0x81459cc (0x7499 bytes) 2024-09-18T00:57:49.811 INFO:teuthology.orchestra.run.smithi042.stdout:315 writesame 0x5154a81 thru 0x515a00b data_size 0x1(0x558b bytes) 2024-09-18T00:57:49.825 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:57:49.826 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:57:49.829 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:49.829 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:49.839 INFO:teuthology.orchestra.run.smithi042.stdout:316 write 0x4e082d3 thru 0x4e13147 (0xae75 bytes) 2024-09-18T00:57:49.841 INFO:teuthology.orchestra.run.smithi042.stdout:317 read 0x400caf4 thru 0x400d217 (0x724 bytes) 2024-09-18T00:57:49.845 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:57:49.847 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:57:49.847 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:57:49.852 INFO:teuthology.orchestra.run.smithi042.stdout:318 write 0xc003c11 thru 0xc00e7de (0xabce bytes) 2024-09-18T00:57:49.857 INFO:teuthology.orchestra.run.smithi042.stdout:319 punch from 0x4b2c647 to 0x4b2fcdc, (0x3695 bytes) 2024-09-18T00:57:49.857 INFO:teuthology.orchestra.run.smithi042.stdout:320 read 0x76f9018 thru 0x7705110 (0xc0f9 bytes) 2024-09-18T00:57:49.858 INFO:teuthology.orchestra.run.smithi042.stdout:321 read 0x999599 thru 0x9a8123 (0xeb8b bytes) 2024-09-18T00:57:49.859 INFO:teuthology.orchestra.run.smithi042.stdout:323 read 0x2c16a8d thru 0x2c18ebd (0x2431 bytes) 2024-09-18T00:57:49.860 INFO:teuthology.orchestra.run.smithi042.stdout:324 read 0xa780cd9 thru 0xa7845e6 (0x390e bytes) 2024-09-18T00:57:49.861 INFO:teuthology.orchestra.run.smithi042.stdout:325 writesame 0x2dd6fc5 thru 0x2ddf0f1 data_size 0x3(0x812d bytes) 2024-09-18T00:57:49.876 INFO:teuthology.orchestra.run.smithi042.stdout:326 write 0xb81e26a thru 0xb828199 (0x9f30 bytes) 2024-09-18T00:57:49.881 INFO:teuthology.orchestra.run.smithi042.stdout:328 writesame 0x1c2f8cc thru 0x1c33873 data_size 0x54(0x3fa8 bytes) 2024-09-18T00:57:49.885 INFO:teuthology.orchestra.run.smithi042.stdout:329 write 0x4b74562 thru 0x4b7bc54 (0x76f3 bytes) 2024-09-18T00:57:49.888 INFO:teuthology.orchestra.run.smithi042.stdout:330 write 0x68c8fa3 thru 0x68ce586 (0x55e4 bytes) 2024-09-18T00:57:49.890 INFO:teuthology.orchestra.run.smithi042.stdout:331 write 0x819c8f5 thru 0x81a1148 (0x4854 bytes) 2024-09-18T00:57:49.893 INFO:teuthology.orchestra.run.smithi042.stdout:335 write 0x80550ac thru 0x805c8e8 (0x783d bytes) 2024-09-18T00:57:49.893 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:49.890+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:49.895 INFO:teuthology.orchestra.run.smithi042.stdout:336 write 0x8470b42 thru 0x8477e5b (0x731a bytes) 2024-09-18T00:57:49.898 INFO:teuthology.orchestra.run.smithi042.stdout:338 writesame 0xa143b84 thru 0xa146ac5 data_size 0x17(0x2f42 bytes) 2024-09-18T00:57:49.901 INFO:teuthology.orchestra.run.smithi042.stdout:339 write 0x6a992c4 thru 0x6a9e1e3 (0x4f20 bytes) 2024-09-18T00:57:49.902 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:49.902 INFO:teuthology.orchestra.run.smithi042.stdout:341 write 0x4916c68 thru 0x491dd73 (0x710c bytes) 2024-09-18T00:57:49.905 INFO:teuthology.orchestra.run.smithi042.stdout:342 read 0x1c49623 thru 0x1c52ada (0x94b8 bytes) 2024-09-18T00:57:49.913 INFO:teuthology.orchestra.run.smithi042.stdout:343 write 0xccfbf88 thru 0xcd07b6e (0xbbe7 bytes) 2024-09-18T00:57:49.916 INFO:teuthology.orchestra.run.smithi042.stdout:344 writesame 0x5d5c616 thru 0x5d6be0c data_size 0x13(0xf7f7 bytes) 2024-09-18T00:57:49.922 INFO:teuthology.orchestra.run.smithi042.stdout:345 read 0xb4fc513 thru 0xb5071d3 (0xacc1 bytes) 2024-09-18T00:57:49.922 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:49.923 INFO:teuthology.orchestra.run.smithi042.stdout:348 writesame 0xacbaddb thru 0xacc9e0a data_size 0x48(0xf030 bytes) 2024-09-18T00:57:49.931 INFO:teuthology.orchestra.run.smithi042.stdout:353 write 0xcfac96c thru 0xcfb5b11 (0x91a6 bytes) 2024-09-18T00:57:49.933 INFO:teuthology.orchestra.run.smithi042.stdout:354 read 0x10004f6 thru 0x100954e (0x9059 bytes) 2024-09-18T00:57:49.936 INFO:teuthology.orchestra.run.smithi042.stdout:355 punch from 0x43180b7 to 0x431bb88, (0x3ad1 bytes) 2024-09-18T00:57:49.936 INFO:teuthology.orchestra.run.smithi042.stdout:356 writesame 0xa6aacd6 thru 0xa6af614 data_size 0x1(0x493f bytes) 2024-09-18T00:57:49.943 INFO:teuthology.orchestra.run.smithi042.stdout:358 write 0x2bc7235 thru 0x2bd3d89 (0xcb55 bytes) 2024-09-18T00:57:49.948 INFO:teuthology.orchestra.run.smithi042.stdout:359 punch from 0xabb2b65 to 0xabb2ddb, (0x276 bytes) 2024-09-18T00:57:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:361 trunc from 0xcfb5b12 to 0xd81650a 2024-09-18T00:57:49.955 INFO:teuthology.orchestra.run.smithi042.stdout:362 write 0x616abab thru 0x616bb25 (0xf7b bytes) 2024-09-18T00:57:49.958 INFO:teuthology.orchestra.run.smithi042.stdout:366 read 0x71dff7 thru 0x72a162 (0xc16c bytes) 2024-09-18T00:57:49.960 INFO:teuthology.orchestra.run.smithi042.stdout:367 write 0xa98dfe3 thru 0xa990f6a (0x2f88 bytes) 2024-09-18T00:57:49.974 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:49.974 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:49.975 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:57:49.985 INFO:teuthology.orchestra.run.smithi042.stdout:368 trunc from 0xd81650a to 0x2f6591d 2024-09-18T00:57:49.994 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:49.990+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:49.997 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:57:50.006 INFO:teuthology.orchestra.run.smithi042.stdout:370 read 0x2139bee thru 0x214043c (0x684f bytes) 2024-09-18T00:57:50.019 INFO:teuthology.orchestra.run.smithi042.stdout:371 trunc from 0x2f6591d to 0xad8b155 2024-09-18T00:57:50.025 INFO:teuthology.orchestra.run.smithi042.stdout:373 read 0x2479b42 thru 0x248901b (0xf4da bytes) 2024-09-18T00:57:50.039 INFO:teuthology.orchestra.run.smithi042.stdout:374 writesame 0xee56538 thru 0xee5f6e4 data_size 0x3(0x91ad bytes) 2024-09-18T00:57:50.039 INFO:teuthology.orchestra.run.smithi042.stdout:375 punch from 0x2ef46a3 to 0x2efcb83, (0x84e0 bytes) 2024-09-18T00:57:50.039 INFO:teuthology.orchestra.run.smithi042.stdout:377 read 0x2c50c9e thru 0x2c5582f (0x4b92 bytes) 2024-09-18T00:57:50.040 INFO:teuthology.orchestra.run.smithi042.stdout:379 writesame 0x9b3e82c thru 0x9b45bc3 data_size 0x9(0x7398 bytes) 2024-09-18T00:57:50.046 INFO:teuthology.orchestra.run.smithi042.stdout:381 punch from 0xb2a257a to 0xb2b0740, (0xe1c6 bytes) 2024-09-18T00:57:50.046 INFO:teuthology.orchestra.run.smithi042.stdout:383 write 0x142ec5b thru 0x143cc6c (0xe012 bytes) 2024-09-18T00:57:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:385 trunc from 0xee5f6e5 to 0x98c4a2f 2024-09-18T00:57:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:387 trunc from 0x98c4a2f to 0x7f55199 2024-09-18T00:57:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:388 writesame 0x7529667 thru 0x75304e0 data_size 0x4f(0x6e7a bytes) 2024-09-18T00:57:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:390 trunc from 0x7f55199 to 0x963a68d 2024-09-18T00:57:50.095 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:50.090+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:392 read 0x5a9244e thru 0x5a97293 (0x4e46 bytes) 2024-09-18T00:57:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:393 write 0x5e70bf thru 0x5f6e35 (0xfd77 bytes) 2024-09-18T00:57:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:394 write 0xcbcbd76 thru 0xcbd3d27 (0x7fb2 bytes) 2024-09-18T00:57:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:395 write 0x83ecf1f thru 0x83f58ec (0x89ce bytes) 2024-09-18T00:57:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:396 read 0xb977f17 thru 0xb9818b7 (0x99a1 bytes) 2024-09-18T00:57:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:397 write 0xa432411 thru 0xa433477 (0x1067 bytes) 2024-09-18T00:57:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:399 write 0x2a08aeb thru 0x2a16477 (0xd98d bytes) 2024-09-18T00:57:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:401 read 0x49829a2 thru 0x4988bd8 (0x6237 bytes) 2024-09-18T00:57:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:402 read 0x44037bd thru 0x440e236 (0xaa7a bytes) 2024-09-18T00:57:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:403 writesame 0xb057123 thru 0xb064140 data_size 0x11(0xd01e bytes) 2024-09-18T00:57:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:405 write 0xc66dfa9 thru 0xc678f1c (0xaf74 bytes) 2024-09-18T00:57:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:406 write 0xeccd826 thru 0xecd4110 (0x68eb bytes) 2024-09-18T00:57:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:407 read 0x4bfbfcd thru 0x4c06236 (0xa26a bytes) 2024-09-18T00:57:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:409 read 0xe63011e thru 0xe635fa4 (0x5e87 bytes) 2024-09-18T00:57:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:410 writesame 0x1e3864f thru 0x1e40533 data_size 0x49(0x7ee5 bytes) 2024-09-18T00:57:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:411 read 0x9a191fc thru 0x9a27ef1 (0xecf6 bytes) 2024-09-18T00:57:50.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:57:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:412 trunc from 0xecd4111 to 0xdb5ef 2024-09-18T00:57:50.196 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:50.190+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:50.248 INFO:teuthology.orchestra.run.smithi042.stdout:413 trunc from 0xdb5ef to 0xbc30c01 2024-09-18T00:57:50.270 INFO:teuthology.orchestra.run.smithi042.stdout:414 trunc from 0xbc30c01 to 0x2b8560d 2024-09-18T00:57:50.282 INFO:teuthology.orchestra.run.smithi042.stdout:415 writesame 0x3109e5 thru 0x316150 data_size 0x1e(0x576c bytes) 2024-09-18T00:57:50.292 INFO:teuthology.orchestra.run.smithi042.stdout:417 trunc from 0x2b8560d to 0x5d6e232 2024-09-18T00:57:50.297 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:50.294+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:50.311 INFO:teuthology.orchestra.run.smithi042.stdout:418 write 0x91a0535 thru 0x91afb82 (0xf64e bytes) 2024-09-18T00:57:50.315 INFO:teuthology.orchestra.run.smithi042.stdout:419 punch from 0x8b2bd29 to 0x8b2dc51, (0x1f28 bytes) 2024-09-18T00:57:50.324 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:50.324 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:50.327 INFO:teuthology.orchestra.run.smithi042.stdout:420 trunc from 0x91afb83 to 0x14083c8 2024-09-18T00:57:50.344 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:57:50.361 INFO:teuthology.orchestra.run.smithi042.stdout:421 trunc from 0x14083c8 to 0xdb3994d 2024-09-18T00:57:50.385 INFO:teuthology.orchestra.run.smithi042.stdout:422 trunc from 0xdb3994d to 0xd9249e 2024-09-18T00:57:50.399 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:50.394+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:50.416 INFO:teuthology.orchestra.run.smithi042.stdout:423 write 0xd45cb4a thru 0xd46a9f1 (0xdea8 bytes) 2024-09-18T00:57:50.420 INFO:teuthology.orchestra.run.smithi042.stdout:424 write 0x1f02150 thru 0x1f0efa1 (0xce52 bytes) 2024-09-18T00:57:50.423 INFO:teuthology.orchestra.run.smithi042.stdout:425 read 0x894486f thru 0x8954771 (0xff03 bytes) 2024-09-18T00:57:50.424 INFO:teuthology.orchestra.run.smithi042.stdout:426 read 0xb29857a thru 0xb2a1de6 (0x986d bytes) 2024-09-18T00:57:50.425 INFO:teuthology.orchestra.run.smithi042.stdout:429 trunc from 0xd46a9f2 to 0xc1c51c0 2024-09-18T00:57:50.435 INFO:teuthology.orchestra.run.smithi042.stdout:430 read 0x8ab72f thru 0x8bb040 (0xf912 bytes) 2024-09-18T00:57:50.437 INFO:teuthology.orchestra.run.smithi042.stdout:431 read 0x9b4959f thru 0x9b4dd57 (0x47b9 bytes) 2024-09-18T00:57:50.437 INFO:teuthology.orchestra.run.smithi042.stdout:432 punch from 0xb369d35 to 0xb375656, (0xb921 bytes) 2024-09-18T00:57:50.437 INFO:teuthology.orchestra.run.smithi042.stdout:433 write 0x1865c30 thru 0x187190c (0xbcdd bytes) 2024-09-18T00:57:50.447 INFO:teuthology.orchestra.run.smithi042.stdout:435 write 0xc60ad6d thru 0xc613823 (0x8ab7 bytes) 2024-09-18T00:57:50.449 INFO:teuthology.orchestra.run.smithi042.stdout:436 writesame 0x777adf3 thru 0x777ba8d data_size 0x1(0xc9b bytes) 2024-09-18T00:57:50.456 INFO:teuthology.orchestra.run.smithi042.stdout:437 writesame 0x66cf3e5 thru 0x66d9625 data_size 0x1(0xa241 bytes) 2024-09-18T00:57:50.476 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:50.476 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:50.489 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:57:50.497 INFO:teuthology.orchestra.run.smithi042.stdout:438 read 0x9867d04 thru 0x986cc49 (0x4f46 bytes) 2024-09-18T00:57:50.499 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:50.494+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:50.509 INFO:teuthology.orchestra.run.smithi042.stdout:440 trunc from 0xc613824 to 0x30f43df 2024-09-18T00:57:50.524 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:50.524 INFO:teuthology.orchestra.run.smithi042.stdout:442 read 0xee7b62 thru 0xef6856 (0xecf5 bytes) 2024-09-18T00:57:50.525 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:50.525 INFO:teuthology.orchestra.run.smithi042.stdout:445 write 0x12372fb thru 0x1239f45 (0x2c4b bytes) 2024-09-18T00:57:50.539 INFO:teuthology.orchestra.run.smithi042.stdout:446 write 0x743fad9 thru 0x744bcda (0xc202 bytes) 2024-09-18T00:57:50.543 INFO:teuthology.orchestra.run.smithi042.stdout:448 read 0x463fc80 thru 0x4641009 (0x138a bytes) 2024-09-18T00:57:50.544 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:50.544 INFO:teuthology.orchestra.run.smithi042.stdout:452 punch from 0xf16134 to 0xf19d22, (0x3bee bytes) 2024-09-18T00:57:50.558 INFO:teuthology.orchestra.run.smithi042.stdout:453 write 0xad2c2b8 thru 0xad3afe0 (0xed29 bytes) 2024-09-18T00:57:50.563 INFO:teuthology.orchestra.run.smithi042.stdout:456 write 0x1cc5617 thru 0x1cd440f (0xedf9 bytes) 2024-09-18T00:57:50.566 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:50.567 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:50.567 INFO:teuthology.orchestra.run.smithi042.stdout:461 punch from 0x3fe17c1 to 0x3ff0eb8, (0xf6f7 bytes) 2024-09-18T00:57:50.567 INFO:teuthology.orchestra.run.smithi042.stdout:465 read 0x21b3648 thru 0x21bcd65 (0x971e bytes) 2024-09-18T00:57:50.567 INFO:teuthology.orchestra.run.smithi042.stdout:466 write 0x517d034 thru 0x5181ffb (0x4fc8 bytes) 2024-09-18T00:57:50.573 INFO:teuthology.orchestra.run.smithi042.stdout:467 punch from 0x822cc60 to 0x822fe1f, (0x31bf bytes) 2024-09-18T00:57:50.573 INFO:teuthology.orchestra.run.smithi042.stdout:468 read 0x53a8d2d thru 0x53ae709 (0x59dd bytes) 2024-09-18T00:57:50.573 INFO:teuthology.orchestra.run.smithi042.stdout:469 write 0x95d842 thru 0x95e012 (0x7d1 bytes) 2024-09-18T00:57:50.590 INFO:teuthology.orchestra.run.smithi042.stdout:472 trunc from 0xad3afe1 to 0x2c476e6 2024-09-18T00:57:50.599 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:50.594+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:50.623 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:50.623 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:50.627 INFO:teuthology.orchestra.run.smithi042.stdout:473 write 0xa8d4382 thru 0xa8e1f0e (0xdb8d bytes) 2024-09-18T00:57:50.638 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:57:50.641 INFO:teuthology.orchestra.run.smithi042.stdout:475 trunc from 0xa8e1f0f to 0x64ff455 2024-09-18T00:57:50.659 INFO:teuthology.orchestra.run.smithi042.stdout:477 punch from 0x26b6d83 to 0x26c284e, (0xbacb bytes) 2024-09-18T00:57:50.660 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:50.660 INFO:teuthology.orchestra.run.smithi042.stdout:479 write 0x29543c0 thru 0x295fb7f (0xb7c0 bytes) 2024-09-18T00:57:50.662 INFO:teuthology.orchestra.run.smithi042.stdout:480 read 0x19cbe21 thru 0x19da1b1 (0xe391 bytes) 2024-09-18T00:57:50.673 INFO:teuthology.orchestra.run.smithi042.stdout:482 trunc from 0x64ff455 to 0xfec7e5 2024-09-18T00:57:50.705 INFO:teuthology.orchestra.run.smithi042.stdout:483 trunc from 0xfec7e5 to 0xa12af62 2024-09-18T00:57:50.716 INFO:teuthology.orchestra.run.smithi042.stdout:485 write 0x447c1e9 thru 0x4488ba3 (0xc9bb bytes) 2024-09-18T00:57:50.723 INFO:teuthology.orchestra.run.smithi042.stdout:486 read 0x38be9aa thru 0x38c0028 (0x167f bytes) 2024-09-18T00:57:50.729 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:50.726+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:50.737 INFO:teuthology.orchestra.run.smithi042.stdout:488 write 0xaf0190d thru 0xaf02c40 (0x1334 bytes) 2024-09-18T00:57:50.743 INFO:teuthology.orchestra.run.smithi042.stdout:490 read 0x9cde65c thru 0x9ce9053 (0xa9f8 bytes) 2024-09-18T00:57:50.743 INFO:teuthology.orchestra.run.smithi042.stdout:491 write 0xc584ae thru 0xc58c19 (0x76c bytes) 2024-09-18T00:57:50.759 INFO:teuthology.orchestra.run.smithi042.stdout:492 writesame 0xb809740 thru 0xb80ca89 data_size 0xd(0x334a bytes) 2024-09-18T00:57:50.759 INFO:teuthology.orchestra.run.smithi042.stdout:493 punch from 0x168e5ac to 0x169df28, (0xf97c bytes) 2024-09-18T00:57:50.759 INFO:teuthology.orchestra.run.smithi042.stdout:495 read 0x94cf8 thru 0x9fc26 (0xaf2f bytes) 2024-09-18T00:57:50.760 INFO:teuthology.orchestra.run.smithi042.stdout:496 write 0xabe7c9c thru 0xabf6c36 (0xef9b bytes) 2024-09-18T00:57:50.764 INFO:teuthology.orchestra.run.smithi042.stdout:498 read 0x1daa413 thru 0x1db7b57 (0xd745 bytes) 2024-09-18T00:57:50.765 INFO:teuthology.orchestra.run.smithi042.stdout:499 punch from 0x9e25e79 to 0x9e321dd, (0xc364 bytes) 2024-09-18T00:57:50.765 INFO:teuthology.orchestra.run.smithi042.stdout:501 read 0x1e20f2b thru 0x1e2f85f (0xe935 bytes) 2024-09-18T00:57:50.766 INFO:teuthology.orchestra.run.smithi042.stdout:502 write 0x963abb5 thru 0x963f2a2 (0x46ee bytes) 2024-09-18T00:57:50.773 INFO:teuthology.orchestra.run.smithi042.stdout:503 write 0x2d066a5 thru 0x2d13cf8 (0xd654 bytes) 2024-09-18T00:57:50.776 INFO:teuthology.orchestra.run.smithi042.stdout:504 read 0xb781e8a thru 0xb78ad24 (0x8e9b bytes) 2024-09-18T00:57:50.776 INFO:teuthology.orchestra.run.smithi042.stdout:505 clone 6 order 21 su 524288 sc 3 2024-09-18T00:57:50.802 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:50.798+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:50.807 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:50.808 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:50.822 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:57:50.910 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:50.906+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:50.969 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:57:50.990 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:50.990 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:51.005 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:57:51.005 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:50.998+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:51.105 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:51.102+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:51.168 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:51.168 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:51.182 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:57:51.207 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:51.202+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:51.301 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:51.301 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:51.304 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-18T00:57:51.308 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:51.302+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:51.316 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:57:51.414 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:51.402+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:51.463 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:51.463 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:51.477 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:57:51.510 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:51.506+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:51.521 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:57:51.521 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:57:51.522 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-18T00:57:51.522 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T00:57:51.522 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T00:57:51.522 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-18T00:57:51.594 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:51.594 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:51.609 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:57:51.610 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:51.606+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:51.638 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:51.638 INFO:teuthology.orchestra.run.smithi195.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T00:57:51.638 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:51.711 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:51.706+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:51.763 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:51.763 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:51.784 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:57:51.826 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:51.822+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:51.870 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:57:51.914 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:51.910+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:51.977 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:51.977 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:51.992 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:57:52.038 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:52.034+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:52.047 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:57:52.047 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:57:52.065 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:57:52.065 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:57:52.082 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:57:52.082 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:57:52.098 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:57:52.099 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:57:52.110 INFO:teuthology.orchestra.run.smithi042.stdout:507 read 0x46fa11a thru 0x470931a (0xf201 bytes) 2024-09-18T00:57:52.114 INFO:teuthology.orchestra.run.smithi042.stdout:508 writesame 0x4448e29 thru 0x4453316 data_size 0x5d(0xa4ee bytes) 2024-09-18T00:57:52.115 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:52.110+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:52.119 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:57:52.120 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:57:52.120 INFO:teuthology.orchestra.run.smithi042.stdout:509 read 0x3f583e6 thru 0x3f648c5 (0xc4e0 bytes) 2024-09-18T00:57:52.131 INFO:teuthology.orchestra.run.smithi042.stdout:510 write 0xe663ace thru 0xe66da7b (0x9fae bytes) 2024-09-18T00:57:52.135 INFO:teuthology.orchestra.run.smithi042.stdout:513 write 0x8c0b6f1 thru 0x8c1789e (0xc1ae bytes) 2024-09-18T00:57:52.139 INFO:teuthology.orchestra.run.smithi042.stdout:514 write 0xafaf301 thru 0xafb09f1 (0x16f1 bytes) 2024-09-18T00:57:52.144 INFO:teuthology.orchestra.run.smithi042.stdout:515 write 0x9666318 thru 0x96740d8 (0xddc1 bytes) 2024-09-18T00:57:52.146 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:57:52.147 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:57:52.148 INFO:teuthology.orchestra.run.smithi042.stdout:518 read 0x4de81a1 thru 0x4df3ecd (0xbd2d bytes) 2024-09-18T00:57:52.156 INFO:teuthology.orchestra.run.smithi042.stdout:519 trunc from 0xe66da7c to 0x8c08588 2024-09-18T00:57:52.159 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:52.159 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:52.174 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:57:52.175 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:57:52.175 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:57:52.175 INFO:teuthology.orchestra.run.smithi042.stdout:520 writesame 0x7a3227e thru 0x7a355af data_size 0x1(0x3332 bytes) 2024-09-18T00:57:52.184 INFO:teuthology.orchestra.run.smithi042.stdout:525 read 0x682d062 thru 0x6839e27 (0xcdc6 bytes) 2024-09-18T00:57:52.184 INFO:teuthology.orchestra.run.smithi042.stdout:526 read 0x6312c3 thru 0x63237c (0x10ba bytes) 2024-09-18T00:57:52.185 INFO:teuthology.orchestra.run.smithi042.stdout:527 write 0x85b9968 thru 0x85c7911 (0xdfaa bytes) 2024-09-18T00:57:52.190 INFO:teuthology.orchestra.run.smithi042.stdout:528 punch from 0x369f713 to 0x36a5e31, (0x671e bytes) 2024-09-18T00:57:52.190 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.191 INFO:teuthology.orchestra.run.smithi042.stdout:530 write 0x71ba97b thru 0x71c6c66 (0xc2ec bytes) 2024-09-18T00:57:52.194 INFO:teuthology.orchestra.run.smithi042.stdout:531 read 0x6aad095 thru 0x6ab113a (0x40a6 bytes) 2024-09-18T00:57:52.196 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:57:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:57:52.202 INFO:teuthology.orchestra.run.smithi042.stdout:532 trunc from 0x8c08588 to 0x3ba9532 2024-09-18T00:57:52.219 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.221 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:52.218+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:52.225 INFO:teuthology.orchestra.run.smithi042.stdout:534 write 0x45a829f thru 0x45acbfe (0x4960 bytes) 2024-09-18T00:57:52.227 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.240 INFO:teuthology.orchestra.run.smithi042.stdout:540 write 0x9881caf thru 0x988cfaa (0xb2fc bytes) 2024-09-18T00:57:52.256 INFO:teuthology.orchestra.run.smithi042.stdout:541 trunc from 0x988cfab to 0x376533b 2024-09-18T00:57:52.269 INFO:teuthology.orchestra.run.smithi042.stdout:542 read 0x2b902ce thru 0x2b949b3 (0x46e6 bytes) 2024-09-18T00:57:52.273 INFO:teuthology.orchestra.run.smithi042.stdout:543 trunc from 0x376533b to 0x577cdf7 2024-09-18T00:57:52.300 INFO:teuthology.orchestra.run.smithi042.stdout:544 write 0xdc56acf thru 0xdc642a2 (0xd7d4 bytes) 2024-09-18T00:57:52.302 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:52.302 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:57:52.302 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:57:52.303 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:57:52.303 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:52.304 INFO:teuthology.orchestra.run.smithi042.stdout:545 writesame 0x298e7e2 thru 0x298fd05 data_size 0x29(0x1524 bytes) 2024-09-18T00:57:52.309 INFO:teuthology.orchestra.run.smithi042.stdout:546 writesame 0x6e19dd5 thru 0x6e1c67e data_size 0xf(0x28aa bytes) 2024-09-18T00:57:52.313 INFO:teuthology.orchestra.run.smithi042.stdout:550 read 0x55cd7cd thru 0x55d4945 (0x7179 bytes) 2024-09-18T00:57:52.313 INFO:teuthology.orchestra.run.smithi042.stdout:551 write 0x6554fd0 thru 0x6560a34 (0xba65 bytes) 2024-09-18T00:57:52.316 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:52.310+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:52.317 INFO:teuthology.orchestra.run.smithi042.stdout:552 write 0xd9932e5 thru 0xd996f01 (0x3c1d bytes) 2024-09-18T00:57:52.318 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:57:52.323 INFO:teuthology.orchestra.run.smithi042.stdout:553 trunc from 0xdc642a3 to 0xc9dd5ec 2024-09-18T00:57:52.335 INFO:teuthology.orchestra.run.smithi042.stdout:554 read 0x1930c9 thru 0x193ca4 (0xbdc bytes) 2024-09-18T00:57:52.337 INFO:teuthology.orchestra.run.smithi042.stdout:555 write 0xb5aa59d thru 0xb5b891e (0xe382 bytes) 2024-09-18T00:57:52.341 INFO:teuthology.orchestra.run.smithi042.stdout:556 read 0x5acfdb thru 0x5b0486 (0x34ac bytes) 2024-09-18T00:57:52.342 INFO:teuthology.orchestra.run.smithi042.stdout:557 punch from 0x40b8f9e to 0x40bd6bd, (0x471f bytes) 2024-09-18T00:57:52.342 INFO:teuthology.orchestra.run.smithi042.stdout:558 write 0x3e2dc43 thru 0x3e3a38a (0xc748 bytes) 2024-09-18T00:57:52.349 INFO:teuthology.orchestra.run.smithi042.stdout:560 read 0x4f1ca07 thru 0x4f2bb0e (0xf108 bytes) 2024-09-18T00:57:52.349 INFO:teuthology.orchestra.run.smithi042.stdout:561 read 0x4d4048a thru 0x4d4ae2a (0xa9a1 bytes) 2024-09-18T00:57:52.349 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.349 INFO:teuthology.orchestra.run.smithi042.stdout:563 write 0xbf44f48 thru 0xbf467d6 (0x188f bytes) 2024-09-18T00:57:52.354 INFO:teuthology.orchestra.run.smithi042.stdout:564 writesame 0x7d5bd8d thru 0x7d67e20 data_size 0x91(0xc094 bytes) 2024-09-18T00:57:52.377 INFO:teuthology.orchestra.run.smithi042.stdout:565 trunc from 0xc9dd5ec to 0x25b0d01 2024-09-18T00:57:52.407 INFO:teuthology.orchestra.run.smithi042.stdout:566 read 0x1293989 thru 0x129620f (0x2887 bytes) 2024-09-18T00:57:52.408 INFO:teuthology.orchestra.run.smithi042.stdout:568 punch from 0x19f4afe to 0x19fa0b5, (0x55b7 bytes) 2024-09-18T00:57:52.417 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:52.410+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:52.427 INFO:teuthology.orchestra.run.smithi042.stdout:570 writesame 0x4aad234 thru 0x4abaa97 data_size 0x16(0xd864 bytes) 2024-09-18T00:57:52.427 INFO:teuthology.orchestra.run.smithi042.stdout:571 read 0x1e41897 thru 0x1e4654c (0x4cb6 bytes) 2024-09-18T00:57:52.428 INFO:teuthology.orchestra.run.smithi042.stdout:572 read 0x1c10837 thru 0x1c115ed (0xdb7 bytes) 2024-09-18T00:57:52.429 INFO:teuthology.orchestra.run.smithi042.stdout:573 write 0x34a0704 thru 0x34a21c7 (0x1ac4 bytes) 2024-09-18T00:57:52.458 INFO:teuthology.orchestra.run.smithi042.stdout:574 writesame 0x7aed233 thru 0x7afb4f3 data_size 0x1(0xe2c1 bytes) 2024-09-18T00:57:52.458 INFO:teuthology.orchestra.run.smithi042.stdout:576 read 0x32c09a5 thru 0x32c5523 (0x4b7f bytes) 2024-09-18T00:57:52.478 INFO:teuthology.orchestra.run.smithi042.stdout:577 write 0xa93117d thru 0xa9315be (0x442 bytes) 2024-09-18T00:57:52.486 INFO:teuthology.orchestra.run.smithi042.stdout:579 read 0x997c965 thru 0x99804cd (0x3b69 bytes) 2024-09-18T00:57:52.486 INFO:teuthology.orchestra.run.smithi042.stdout:580 read 0x7141c86 thru 0x714e04a (0xc3c5 bytes) 2024-09-18T00:57:52.486 INFO:teuthology.orchestra.run.smithi042.stdout:581 read 0x708ba24 thru 0x7091a4e (0x602b bytes) 2024-09-18T00:57:52.492 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:52.492 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:57:52.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:57:52.492 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:57:52.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.492 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4041 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:52.492 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.4854641810000002, 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037088000000000003, 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4041 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003005+0000", 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.187e-06 2024-09-18T00:57:52.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003011+0000", 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003029+0000", 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8026999999999999e-05 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003043+0000", 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039484+0000", 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036440895000000001 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039722+0000", 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000238507 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040092+0000", 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000369818 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.4854065580000002, 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.497 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:57:52.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:57:52.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.4853483010000001, 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.4852898620000001, 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:57:52.506 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:57:52.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.510 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.4852286450000003, 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.511 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:57:52.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:57:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.4851382300000004, 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:57:52.515 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:57:52.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.519 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.4850672969999996, 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.520 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:57:52.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.524 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.4850076909999999, 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:57:52.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:57:52.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.4849483210000001, 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.529 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.533 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.484886532, 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:57:52.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:57:52.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.4847537089999996, 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:57:52.538 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:57:52.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.542 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9295362320000002, 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.543 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T00:57:52.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T00:57:52.548 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8459 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559110+0000", 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9293586559999998, 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037081250000000003, 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.549 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8459 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559110+0000", 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559110+0000", 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559110+0000", 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2099999999999996e-07 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559123+0000", 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2809000000000001e-05 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559130+0000", 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2339999999999997e-06 2024-09-18T00:57:52.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559364+0000", 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023426299999999999 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596092+0000", 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036727965000000001 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596146+0000", 2024-09-18T00:57:52.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3372000000000003e-05 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596191+0000", 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4985999999999999e-05 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:57:52.553 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.929036108, 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:57:52.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T00:57:52.557 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9290056, 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T00:57:52.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9289809350000002, 2024-09-18T00:57:52.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:57:52.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:57:52.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T00:57:52.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9289524030000003, 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T00:57:52.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T00:57:52.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9286191119999998, 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T00:57:52.571 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T00:57:52.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.575 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.928577261, 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T00:57:52.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T00:57:52.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T00:57:52.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T00:57:52.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9275397650000001, 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.582 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:57:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:57:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:52.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:52.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:52.586 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:52.588 INFO:teuthology.orchestra.run.smithi042.stdout:582 trunc from 0xa9315bf to 0x78cdf6e 2024-09-18T00:57:52.588 INFO:teuthology.orchestra.run.smithi042.stdout:584 read 0x2a5790a thru 0x2a65dc0 (0xe4b7 bytes) 2024-09-18T00:57:52.588 INFO:teuthology.orchestra.run.smithi042.stdout:586 read 0x6615053 thru 0x6617b69 (0x2b17 bytes) 2024-09-18T00:57:52.589 INFO:teuthology.orchestra.run.smithi042.stdout:590 write 0x8729ab4 thru 0x873510c (0xb659 bytes) 2024-09-18T00:57:52.589 INFO:teuthology.orchestra.run.smithi042.stdout:591 read 0x584780f thru 0x58497a0 (0x1f92 bytes) 2024-09-18T00:57:52.589 INFO:teuthology.orchestra.run.smithi042.stdout:592 writesame 0x863a55d thru 0x8641283 data_size 0x1(0x6d27 bytes) 2024-09-18T00:57:52.589 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:52.582+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:52.589 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:57:52.590 INFO:teuthology.orchestra.run.smithi042.stdout:593 trunc from 0x873510d to 0xde22fc6 2024-09-18T00:57:52.602 INFO:teuthology.orchestra.run.smithi042.stdout:594 punch from 0x55e41e to 0x56cbb3, (0xe795 bytes) 2024-09-18T00:57:52.603 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xe9e7e19 2024-09-18T00:57:52.604 INFO:teuthology.orchestra.run.smithi042.stdout:596 trunc from 0xde22fc6 to 0xe9e7e19 2024-09-18T00:57:52.616 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.616 INFO:teuthology.orchestra.run.smithi042.stdout:598 write 0xce4328a thru 0xce507cf (0xd546 bytes) 2024-09-18T00:57:52.622 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.622 INFO:teuthology.orchestra.run.smithi042.stdout:600 read 0x279bab1 thru 0x27a3a39 (0x7f89 bytes) 2024-09-18T00:57:52.625 INFO:teuthology.orchestra.run.smithi042.stdout:601 trunc from 0xe9e7e19 to 0xdee61db 2024-09-18T00:57:52.645 INFO:teuthology.orchestra.run.smithi042.stdout:602 trunc from 0xdee61db to 0x9c4dba8 2024-09-18T00:57:52.662 INFO:teuthology.orchestra.run.smithi042.stdout:603 writesame 0x152fb8a thru 0x153e8e7 data_size 0x17(0xed5e bytes) 2024-09-18T00:57:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.672 INFO:teuthology.orchestra.run.smithi042.stdout:606 punch from 0x61d974a to 0x61e70c8, (0xd97e bytes) 2024-09-18T00:57:52.673 INFO:teuthology.orchestra.run.smithi042.stdout:608 writesame 0xf5ac97 thru 0xf5c29c data_size 0x1(0x1606 bytes) 2024-09-18T00:57:52.687 INFO:teuthology.orchestra.run.smithi042.stdout:609 read 0x3c0ef5d thru 0x3c18a45 (0x9ae9 bytes) 2024-09-18T00:57:52.687 INFO:teuthology.orchestra.run.smithi042.stdout:610 write 0x3855fde thru 0x3863a6a (0xda8d bytes) 2024-09-18T00:57:52.688 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:52.682+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:52.694 INFO:teuthology.orchestra.run.smithi042.stdout:612 writesame 0x110d7bc thru 0x1119294 data_size 0x1(0xbad9 bytes) 2024-09-18T00:57:52.730 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:52.730 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:52.731 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:52.731 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:52.745 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:57:52.762 INFO:teuthology.orchestra.run.smithi042.stdout:613 read 0x52e50ca thru 0x52e6804 (0x173b bytes) 2024-09-18T00:57:52.775 INFO:teuthology.orchestra.run.smithi042.stdout:614 writesame 0xcd5a596 thru 0xcd69a8e data_size 0x77(0xf4f9 bytes) 2024-09-18T00:57:52.776 INFO:teuthology.orchestra.run.smithi042.stdout:615 read 0x437fcd9 thru 0x438aeea (0xb212 bytes) 2024-09-18T00:57:52.776 INFO:teuthology.orchestra.run.smithi042.stdout:616 flatten 2024-09-18T00:57:52.789 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:52.786+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:617 read 0x509701a thru 0x50a12bb (0xa2a2 bytes) 2024-09-18T00:57:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:618 read 0x80efb92 thru 0x80fc355 (0xc7c4 bytes) 2024-09-18T00:57:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:620 punch from 0xb26972e to 0xb269af8, (0x3ca bytes) 2024-09-18T00:57:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.795 INFO:teuthology.orchestra.run.smithi042.stdout:622 writesame 0x29121f4 thru 0x2920f11 data_size 0x43(0xed1e bytes) 2024-09-18T00:57:52.798 INFO:teuthology.orchestra.run.smithi042.stdout:623 writesame 0x6c4b805 thru 0x6c58fd7 data_size 0x15(0xd7d3 bytes) 2024-09-18T00:57:52.806 INFO:teuthology.orchestra.run.smithi042.stdout:624 punch from 0x12d2971 to 0x12d863a, (0x5cc9 bytes) 2024-09-18T00:57:52.806 INFO:teuthology.orchestra.run.smithi042.stdout:625 write 0x19c84ed thru 0x19d4145 (0xbc59 bytes) 2024-09-18T00:57:52.812 INFO:teuthology.orchestra.run.smithi042.stdout:626 writesame 0x341c11c thru 0x3429f86 data_size 0x1(0xde6b bytes) 2024-09-18T00:57:52.861 INFO:teuthology.orchestra.run.smithi042.stdout:627 write 0xa23e68e thru 0xa24265a (0x3fcd bytes) 2024-09-18T00:57:52.867 INFO:teuthology.orchestra.run.smithi042.stdout:628 read 0x80654ce thru 0x80680aa (0x2bdd bytes) 2024-09-18T00:57:52.867 INFO:teuthology.orchestra.run.smithi042.stdout:629 write 0x8edc62a thru 0x8ee4aa5 (0x847c bytes) 2024-09-18T00:57:52.872 INFO:teuthology.orchestra.run.smithi042.stdout:631 write 0x2e8466c thru 0x2e89c48 (0x55dd bytes) 2024-09-18T00:57:52.878 INFO:teuthology.orchestra.run.smithi042.stdout:632 read 0x3c97f69 thru 0x3c9fcc5 (0x7d5d bytes) 2024-09-18T00:57:52.878 INFO:teuthology.orchestra.run.smithi042.stdout:635 writesame 0xbcd656e thru 0xbcda022 data_size 0x7(0x3ab5 bytes) 2024-09-18T00:57:52.890 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:52.886+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:52.891 INFO:teuthology.orchestra.run.smithi042.stdout:637 trunc from 0xcd69a8f to 0x8c535cb 2024-09-18T00:57:52.899 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:52.899 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:52.899 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:52.899 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:52.899 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:52.912 INFO:teuthology.orchestra.run.smithi042.stdout:639 read 0x62eabf1 thru 0x62f434b (0x975b bytes) 2024-09-18T00:57:52.912 INFO:teuthology.orchestra.run.smithi042.stdout:643 punch from 0x5e737bc to 0x5e74b3d, (0x1381 bytes) 2024-09-18T00:57:52.912 INFO:teuthology.orchestra.run.smithi042.stdout:645 read 0x276587a thru 0x276e81d (0x8fa4 bytes) 2024-09-18T00:57:52.912 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.913 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:57:52.925 INFO:teuthology.orchestra.run.smithi042.stdout:649 trunc from 0x8c535cb to 0x10e188b 2024-09-18T00:57:52.947 INFO:teuthology.orchestra.run.smithi042.stdout:650 read 0xa08cda thru 0xa0a252 (0x1579 bytes) 2024-09-18T00:57:52.947 INFO:teuthology.orchestra.run.smithi042.stdout:651 read 0xc14b67 thru 0xc22e43 (0xe2dd bytes) 2024-09-18T00:57:52.948 INFO:teuthology.orchestra.run.smithi042.stdout:652 read 0xc91b00 thru 0xca1759 (0xfc5a bytes) 2024-09-18T00:57:52.969 INFO:teuthology.orchestra.run.smithi042.stdout:654 writesame 0x9401f59 thru 0x9406921 data_size 0x1(0x49c9 bytes) 2024-09-18T00:57:52.969 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.970 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:52.970 INFO:teuthology.orchestra.run.smithi042.stdout:657 write 0x5c26e3f thru 0x5c345b8 (0xd77a bytes) 2024-09-18T00:57:52.975 INFO:teuthology.orchestra.run.smithi042.stdout:658 punch from 0x67f1db2 to 0x67f993a, (0x7b88 bytes) 2024-09-18T00:57:52.976 INFO:teuthology.orchestra.run.smithi042.stdout:660 trunc from 0x9406922 to 0x8a35ba8 2024-09-18T00:57:52.984 INFO:teuthology.orchestra.run.smithi042.stdout:661 read 0x3977ea6 thru 0x3981140 (0x929b bytes) 2024-09-18T00:57:52.984 INFO:teuthology.orchestra.run.smithi042.stdout:662 write 0x1347f9f thru 0x1357e79 (0xfedb bytes) 2024-09-18T00:57:52.991 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:52.986+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:52.991 INFO:teuthology.orchestra.run.smithi042.stdout:663 trunc from 0x8a35ba8 to 0xa604a7b 2024-09-18T00:57:52.994 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:57:53.011 INFO:teuthology.orchestra.run.smithi042.stdout:664 write 0xd78cfa2 thru 0xd78d1ba (0x219 bytes) 2024-09-18T00:57:53.016 INFO:teuthology.orchestra.run.smithi042.stdout:665 read 0x1cdc171 thru 0x1ce08f0 (0x4780 bytes) 2024-09-18T00:57:53.016 INFO:teuthology.orchestra.run.smithi042.stdout:666 punch from 0x55b0783 to 0x55b50e3, (0x4960 bytes) 2024-09-18T00:57:53.016 INFO:teuthology.orchestra.run.smithi042.stdout:668 write 0x5bd0024 thru 0x5bd0447 (0x424 bytes) 2024-09-18T00:57:53.021 INFO:teuthology.orchestra.run.smithi042.stdout:669 write 0xc419bc2 thru 0xc423728 (0x9b67 bytes) 2024-09-18T00:57:53.025 INFO:teuthology.orchestra.run.smithi042.stdout:670 write 0xd6ff76c thru 0xd70ed44 (0xf5d9 bytes) 2024-09-18T00:57:53.052 INFO:teuthology.orchestra.run.smithi042.stdout:671 trunc from 0xd78d1bb to 0x91dd9b 2024-09-18T00:57:53.072 INFO:teuthology.orchestra.run.smithi042.stdout:672 read 0x3f29e4 thru 0x3f6ade (0x40fb bytes) 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3710 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2073189670000009, 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024498702000000001, 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3710 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867971+0000", 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.671e-06 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867976+0000", 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7030000000000003e-06 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867981+0000", 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8520000000000003e-06 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892175+0000", 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024193538000000001 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892279+0000", 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104017 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892467+0000", 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018858699999999999 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.082 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3711 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2073006470000003, 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024701471999999999, 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3711 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6199999999999999e-07 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867989+0000", 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867995+0000", 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.127e-06 2024-09-18T00:57:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868000+0000", 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892500+0000", 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024499725999999999 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892587+0000", 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7232999999999996e-05 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892688+0000", 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010115000000000001 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3712 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:53.087 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2072816890000002, 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024914385000000001, 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3712 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868008+0000", 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868014+0000", 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868019+0000", 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892727+0000", 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024707638000000001 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892817+0000", 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9569999999999998e-05 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892920+0000", 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103744 2024-09-18T00:57:53.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3713 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2072621580000007, 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025115858000000001, 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3713 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:57:53.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868026+0000", 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.58e-07 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868032+0000", 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3790000000000003e-06 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868039+0000", 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6969999999999996e-06 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868044+0000", 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7060000000000003e-06 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892954+0000", 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024910704999999998 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893041+0000", 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6747000000000001e-05 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893141+0000", 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010026600000000001 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3714 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2072373509999998, 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025291557999999999, 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3714 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868051+0000", 2024-09-18T00:57:53.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868052+0000", 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.863e-06 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868059+0000", 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2720000000000003e-06 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868064+0000", 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8860000000000003e-06 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893175+0000", 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025111114 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893262+0000", 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7476000000000001e-05 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893342+0000", 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9577000000000001e-05 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3715 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2072178460000007, 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025471191000000001, 2024-09-18T00:57:53.101 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3715 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2799999999999996e-07 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868072+0000", 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9269999999999999e-06 2024-09-18T00:57:53.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868078+0000", 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9830000000000001e-06 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868083+0000", 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.972e-06 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893373+0000", 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025290316 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893463+0000", 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9078000000000003e-05 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893541+0000", 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8386999999999993e-05 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3716 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2071987879999995, 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025664781000000001, 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3716 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2099999999999998e-07 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868091+0000", 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8950000000000001e-06 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868097+0000", 2024-09-18T00:57:53.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1929999999999998e-06 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868102+0000", 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6789999999999996e-06 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893570+0000", 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025468227999999999 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893657+0000", 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7176999999999996e-05 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893754+0000", 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6287999999999997e-05 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3717 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2071796819999996, 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025815932999999999, 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3717 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868111+0000", 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6860000000000002e-06 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868117+0000", 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868122+0000", 2024-09-18T00:57:53.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.69e-06 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893802+0000", 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025680397000000001 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893869+0000", 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6597e-05 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893924+0000", 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5031999999999998e-05 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3718 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2071586599999993, 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026018052, 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3718 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:53.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868131+0000", 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9369999999999998e-06 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868138+0000", 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4389999999999997e-06 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868143+0000", 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1440000000000002e-06 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893973+0000", 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025830490000000001 2024-09-18T00:57:53.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894063+0000", 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9313000000000004e-05 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894147+0000", 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4342e-05 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3719 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2071387040000001, 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026147934000000001, 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3719 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:53.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868151+0000", 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868157+0000", 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868162+0000", 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5619999999999997e-06 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894183+0000", 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026021083 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894241+0000", 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8505999999999999e-05 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894297+0000", 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5396000000000003e-05 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3720 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2071201869999992, 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026376077000000001, 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3720 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868168+0000", 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868170+0000", 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0889999999999998e-06 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868176+0000", 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3090000000000001e-06 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868181+0000", 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8810000000000004e-06 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894341+0000", 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026159636 2024-09-18T00:57:53.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894430+0000", 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8986000000000005e-05 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894543+0000", 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011382700000000001 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3721 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2071006690000008, 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569722, 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3721 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.129 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:53.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868189+0000", 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7460000000000001e-06 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868195+0000", 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8490000000000002e-06 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868200+0000", 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1669999999999998e-06 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894623+0000", 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02642338 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894688+0000", 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4183000000000005e-05 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894757+0000", 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9062999999999999e-05 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3722 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2070819840000002, 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026735041000000001, 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3722 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868208+0000", 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8470000000000001e-06 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868214+0000", 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1090000000000003e-06 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868218+0000", 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.476e-06 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894788+0000", 2024-09-18T00:57:53.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569274 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894868+0000", 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0711999999999996e-05 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894941+0000", 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3723 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2070635680000006, 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026971922999999998, 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3723 2024-09-18T00:57:53.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-07 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868226+0000", 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.812e-06 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868232+0000", 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7910000000000001e-06 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868237+0000", 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7339999999999999e-06 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895056+0000", 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026819195000000001 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895103+0000", 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7264e-05 2024-09-18T00:57:53.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895196+0000", 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2802999999999999e-05 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3724 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2070453560000001, 2024-09-18T00:57:53.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027181833999999998, 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3724 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868243+0000", 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868245+0000", 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1160000000000002e-06 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868251+0000", 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1410000000000002e-06 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868256+0000", 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0130000000000001e-06 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895249+0000", 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02699352 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895338+0000", 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8331000000000005e-05 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895424+0000", 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6348999999999996e-05 2024-09-18T00:57:53.146 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3725 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2070262219999996, 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027296051000000002, 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.147 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3725 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868262+0000", 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868267+0000", 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868274+0000", 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2360000000000001e-06 2024-09-18T00:57:53.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868278+0000", 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7620000000000003e-06 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895458+0000", 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027179608000000001 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895511+0000", 2024-09-18T00:57:53.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2738000000000002e-05 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895557+0000", 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6631999999999997e-05 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3726 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871463+0000", 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2038244070000008, 2024-09-18T00:57:53.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024295104000000001, 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3726 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871463+0000", 2024-09-18T00:57:53.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871463+0000", 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871466+0000", 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8169999999999999e-06 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871480+0000", 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3645999999999999e-05 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871506+0000", 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.637e-05 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871535+0000", 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9249999999999999e-05 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895596+0000", 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024060537999999999 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895678+0000", 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2589999999999994e-05 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895758+0000", 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9893000000000002e-05 2024-09-18T00:57:53.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3727 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2037156709999994, 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024398075000000002, 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.156 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3727 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871573+0000", 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.367e-06 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871580+0000", 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8029999999999999e-06 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871600+0000", 2024-09-18T00:57:53.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0213000000000001e-05 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871615+0000", 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.471e-05 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895805+0000", 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024189677 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895923+0000", 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118496 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895970+0000", 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6808999999999999e-05 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3728 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:53.160 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2036544120000006, 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024637071999999999, 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3728 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871634+0000", 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.074e-06 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871642+0000", 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8830000000000006e-06 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871660+0000", 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8283000000000001e-05 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871692+0000", 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1714000000000002e-05 2024-09-18T00:57:53.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896004+0000", 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024312099 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896099+0000", 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5036000000000003e-05 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896270+0000", 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017098300000000001 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0113225530000003, 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:53.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:57:53.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.170 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.171 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.171 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:53.172 INFO:teuthology.orchestra.run.smithi042.stdout:673 write 0x24e3983 thru 0x24f0cf7 (0xd375 bytes) 2024-09-18T00:57:53.172 INFO:teuthology.orchestra.run.smithi042.stdout:674 read 0x9467f6 thru 0x94c9dc (0x61e7 bytes) 2024-09-18T00:57:53.172 INFO:teuthology.orchestra.run.smithi042.stdout:675 punch from 0x181cf6d to 0x1820e83, (0x3f16 bytes) 2024-09-18T00:57:53.172 INFO:teuthology.orchestra.run.smithi042.stdout:677 punch from 0xb5ea54 to 0xb661f8, (0x77a4 bytes) 2024-09-18T00:57:53.173 INFO:teuthology.orchestra.run.smithi042.stdout:678 punch from 0x3272dd to 0x32d792, (0x64b5 bytes) 2024-09-18T00:57:53.173 INFO:teuthology.orchestra.run.smithi042.stdout:679 read 0x117733 thru 0x11fb91 (0x845f bytes) 2024-09-18T00:57:53.173 INFO:teuthology.orchestra.run.smithi042.stdout:680 read 0x19753a thru 0x19d91c (0x63e3 bytes) 2024-09-18T00:57:53.173 INFO:teuthology.orchestra.run.smithi042.stdout:682 trunc from 0x24f0cf8 to 0x34cc52b 2024-09-18T00:57:53.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:57:53.175 INFO:teuthology.orchestra.run.smithi042.stdout:683 trunc from 0x34cc52b to 0x63213e1 2024-09-18T00:57:53.175 INFO:teuthology.orchestra.run.smithi042.stdout:685 trunc from 0x63213e1 to 0x494629d 2024-09-18T00:57:53.175 INFO:teuthology.orchestra.run.smithi042.stdout:686 read 0x1ed25b5 thru 0x1edab1e (0x856a bytes) 2024-09-18T00:57:53.175 INFO:teuthology.orchestra.run.smithi042.stdout:687 writesame 0xde2457a thru 0xde3167c data_size 0x1(0xd103 bytes) 2024-09-18T00:57:53.175 INFO:teuthology.orchestra.run.smithi042.stdout:688 write 0xb642a8b thru 0xb647f95 (0x550b bytes) 2024-09-18T00:57:53.175 INFO:teuthology.orchestra.run.smithi042.stdout:690 trunc from 0xde3167d to 0x295e4d7 2024-09-18T00:57:53.175 INFO:teuthology.orchestra.run.smithi042.stdout:691 punch from 0x27f9c78 to 0x2802f4b, (0x92d3 bytes) 2024-09-18T00:57:53.175 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:53.176 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:53.166+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:53.181 INFO:teuthology.orchestra.run.smithi042.stdout:693 writesame 0x3792096 thru 0x3797aad data_size 0x4(0x5a18 bytes) 2024-09-18T00:57:53.189 INFO:teuthology.orchestra.run.smithi042.stdout:694 write 0x51bae5c thru 0x51bd805 (0x29aa bytes) 2024-09-18T00:57:53.214 INFO:teuthology.orchestra.run.smithi042.stdout:695 write 0xe454d8b thru 0xe46074f (0xb9c5 bytes) 2024-09-18T00:57:53.220 INFO:teuthology.orchestra.run.smithi042.stdout:696 writesame 0xb33638d thru 0xb33f783 data_size 0x1(0x93f7 bytes) 2024-09-18T00:57:53.248 INFO:teuthology.orchestra.run.smithi042.stdout:699 read 0xa0b2ef3 thru 0xa0b41f9 (0x1307 bytes) 2024-09-18T00:57:53.249 INFO:teuthology.orchestra.run.smithi042.stdout:701 clone 7 order 20 su 65536 sc 4 2024-09-18T00:57:53.273 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:53.270+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:53.305 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:53.305 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:53.305 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:53.305 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:53.320 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:57:53.374 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:53.370+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:53.475 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:53.470+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:53.481 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:53.481 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:53.481 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:53.496 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:57:53.582 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:53.578+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:53.670 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:57:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:57:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.377650241000001, 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:57:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:57:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:57:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:57:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7423 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1813516909999997, 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045296661000000002, 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7423 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485621+0000", 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485627+0000", 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.156e-06 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485633+0000", 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789999999999999e-06 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530788+0000", 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045155001 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530853+0000", 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5634000000000002e-05 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530915+0000", 2024-09-18T00:57:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2022999999999999e-05 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7424 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1146880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1813307010000003, 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045419355000000002, 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7424 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485642+0000", 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0640000000000001e-06 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485649+0000", 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485654+0000", 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2329999999999997e-06 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530941+0000", 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045286602000000002 2024-09-18T00:57:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531004+0000", 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3455999999999997e-05 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531059+0000", 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4771000000000001e-05 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1813091880000002, 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:57:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:57:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1812877510000002, 2024-09-18T00:57:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:57:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:57:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:57:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:57:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:57:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1811924239999998, 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:57:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1811669849999999, 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:57:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.705 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1811444619999998, 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:53.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:57:53.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:53.710 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1811210220000001, 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:53.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:57:53.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1810972120000001, 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:57:53.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1810748469999997, 2024-09-18T00:57:53.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:53.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:57:53.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.18104823, 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.728 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.176896459, 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.729 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:53.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:57:53.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1768757159999996, 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:57:53.733 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:57:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.737 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1768301509999999, 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:57:53.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:57:53.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:53.742 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.17680381, 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:57:53.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1767811400000001, 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:57:53.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1767627669999996, 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:57:53.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.756 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1767429790000001, 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:57:53.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:57:53.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.176716109, 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:53.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:53.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:57:53.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:53.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:53.765 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:53.767 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:57:53.767 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:53.762+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:53.868 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:53.862+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:53.942 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:53.942 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:53.942 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:53.942 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:53.956 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:57:53.969 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:53.962+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:53.990 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:57:54.070 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:54.066+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:54.119 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:54.120 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:54.138 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:57:54.186 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:54.182+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:54.219 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-18T00:57:54.222 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:57:54.272 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:54.266+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:54.338 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:57:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:57:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:57:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.258275782, 2024-09-18T00:57:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:57:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:57:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:57:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.257346191, 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:57:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:57:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:57:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.257332131, 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:57:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:57:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:57:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:57:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.256041556, 2024-09-18T00:57:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:57:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:57:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.256021084, 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:57:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:57:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.253226631, 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:57:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:57:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:57:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.253211447, 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:57:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:57:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.253188501, 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:57:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1096 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:57:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.249026162, 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058813730000000002, 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1096 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:57:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.75e-07 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086206+0000", 2024-09-18T00:57:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.677e-06 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086210+0000", 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5950000000000001e-06 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086214+0000", 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4979999999999998e-06 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.144798+0000", 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058584587 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.144893+0000", 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4413000000000006e-05 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145017+0000", 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000124485 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.249010862, 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:57:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:57:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.248989645, 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.387 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.248968262, 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:57:54.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.245068408, 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:57:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:57:54.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:57:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.245054207000001, 2024-09-18T00:57:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:57:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:57:54.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:57:54.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.245036170000001, 2024-09-18T00:57:54.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:57:54.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:57:54.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.245022104, 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:57:54.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:57:54.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.240996752999999, 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:57:54.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.240982741, 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:57:54.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:57:54.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:57:54.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.240968777000001, 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.419 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:57:54.423 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4791108230000001, 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:57:54.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:54.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:54.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:54.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:57:54.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:57:54.428 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.428 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.428 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.428 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:54.428 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:54.428 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:54.430 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:57:54.430 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:57:54.430 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:57:54.431 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:57:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:57:54.431 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:57:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:57:54.432 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:57:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:57:54.432 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:54.422+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:54.445 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:57:54.445 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:57:54.466 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:57:54.467 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:57:54.489 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:57:54.489 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:57:54.510 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:57:54.511 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:57:54.530 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:54.526+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:54.566 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:54.567 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:54.567 INFO:teuthology.orchestra.run.smithi042.stdout:702 read 0x4dc5e3b thru 0x4dd0c03 (0xadc9 bytes) 2024-09-18T00:57:54.586 INFO:teuthology.orchestra.run.smithi042.stdout:703 trunc from 0xe460750 to 0x4611e20 2024-09-18T00:57:54.591 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:57:54.604 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:54.604 INFO:teuthology.orchestra.run.smithi042.stdout:705 writesame 0xc61020 thru 0xc6b537 data_size 0x24(0xa518 bytes) 2024-09-18T00:57:54.609 INFO:teuthology.orchestra.run.smithi042.stdout:706 read 0x27cec2b thru 0x27d6ba8 (0x7f7e bytes) 2024-09-18T00:57:54.610 INFO:teuthology.orchestra.run.smithi042.stdout:708 read 0x42c768d thru 0x42d2067 (0xa9db bytes) 2024-09-18T00:57:54.623 INFO:teuthology.orchestra.run.smithi042.stdout:709 write 0x8588f7d thru 0x85987e8 (0xf86c bytes) 2024-09-18T00:57:54.629 INFO:teuthology.orchestra.run.smithi042.stdout:710 read 0x1022f9e thru 0x102e1d2 (0xb235 bytes) 2024-09-18T00:57:54.629 INFO:teuthology.orchestra.run.smithi042.stdout:711 read 0x3e4a483 thru 0x3e52936 (0x84b4 bytes) 2024-09-18T00:57:54.629 INFO:teuthology.orchestra.run.smithi042.stdout:714 writesame 0x84460c1 thru 0x84542b5 data_size 0x17(0xe1f5 bytes) 2024-09-18T00:57:54.631 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:54.626+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:54.633 INFO:teuthology.orchestra.run.smithi042.stdout:715 write 0x183b7a6 thru 0x1847fb1 (0xc80c bytes) 2024-09-18T00:57:54.638 INFO:teuthology.orchestra.run.smithi042.stdout:716 write 0x5e87019 thru 0x5e89935 (0x291d bytes) 2024-09-18T00:57:54.641 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:54.649 INFO:teuthology.orchestra.run.smithi042.stdout:718 trunc from 0x85987e9 to 0x2d73642 2024-09-18T00:57:54.670 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:54.682 INFO:teuthology.orchestra.run.smithi042.stdout:720 writesame 0x73924f7 thru 0x739e242 data_size 0xa(0xbd4c bytes) 2024-09-18T00:57:54.692 INFO:teuthology.orchestra.run.smithi042.stdout:721 writesame 0x9ad88c5 thru 0x9ae6a51 data_size 0x13(0xe18d bytes) 2024-09-18T00:57:54.692 INFO:teuthology.orchestra.run.smithi042.stdout:722 punch from 0x58fcc0f to 0x5908e6f, (0xc260 bytes) 2024-09-18T00:57:54.692 INFO:teuthology.orchestra.run.smithi042.stdout:724 punch from 0x3efcd9c to 0x3f0a666, (0xd8ca bytes) 2024-09-18T00:57:54.692 INFO:teuthology.orchestra.run.smithi042.stdout:725 read 0x86916dd thru 0x8693325 (0x1c49 bytes) 2024-09-18T00:57:54.692 INFO:teuthology.orchestra.run.smithi042.stdout:726 writesame 0x2b46b69 thru 0x2b491e7 data_size 0x2d(0x267f bytes) 2024-09-18T00:57:54.709 INFO:teuthology.orchestra.run.smithi042.stdout:727 write 0xd38fd44 thru 0xd395855 (0x5b12 bytes) 2024-09-18T00:57:54.711 INFO:teuthology.orchestra.run.smithi042.stdout:728 writesame 0x3437690 thru 0x34447d1 data_size 0x37(0xd142 bytes) 2024-09-18T00:57:54.718 INFO:teuthology.orchestra.run.smithi042.stdout:729 writesame 0x6d75e4 thru 0x6d85ea data_size 0x1(0x1007 bytes) 2024-09-18T00:57:54.724 INFO:teuthology.orchestra.run.smithi042.stdout:730 punch from 0x374fa4 to 0x3828ed, (0xd949 bytes) 2024-09-18T00:57:54.725 INFO:teuthology.orchestra.run.smithi042.stdout:732 write 0x7dc7cc3 thru 0x7dcd871 (0x5baf bytes) 2024-09-18T00:57:54.729 INFO:teuthology.orchestra.run.smithi042.stdout:733 read 0xd1fa672 thru 0xd207892 (0xd221 bytes) 2024-09-18T00:57:54.732 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:54.726+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:54.744 INFO:teuthology.orchestra.run.smithi042.stdout:735 trunc from 0xd395856 to 0x3503895 2024-09-18T00:57:54.748 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:54.748 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:54.762 INFO:teuthology.orchestra.run.smithi042.stdout:736 writesame 0x9c15db thru 0x9ccefd data_size 0x1(0xb923 bytes) 2024-09-18T00:57:54.767 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:57:54.815 INFO:teuthology.orchestra.run.smithi042.stdout:737 punch from 0x1233f3d to 0x123405e, (0x121 bytes) 2024-09-18T00:57:54.830 INFO:teuthology.orchestra.run.smithi042.stdout:739 write 0x8a38454 thru 0x8a42fc7 (0xab74 bytes) 2024-09-18T00:57:54.833 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:54.826+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:54.836 INFO:teuthology.orchestra.run.smithi042.stdout:741 read 0x894c52a thru 0x895435b (0x7e32 bytes) 2024-09-18T00:57:54.837 INFO:teuthology.orchestra.run.smithi042.stdout:742 read 0x315b283 thru 0x315d1d3 (0x1f51 bytes) 2024-09-18T00:57:54.837 INFO:teuthology.orchestra.run.smithi042.stdout:743 write 0x81f85ff thru 0x81fe12a (0x5b2c bytes) 2024-09-18T00:57:54.841 INFO:teuthology.orchestra.run.smithi042.stdout:744 read 0x1a73ab7 thru 0x1a83491 (0xf9db bytes) 2024-09-18T00:57:54.842 INFO:teuthology.orchestra.run.smithi042.stdout:745 writesame 0x6b0a7cf thru 0x6b19195 data_size 0x1(0xe9c7 bytes) 2024-09-18T00:57:54.882 INFO:teuthology.orchestra.run.smithi042.stdout:748 writesame 0x3d6e7d6 thru 0x3d753da data_size 0x1(0x6c05 bytes) 2024-09-18T00:57:54.897 INFO:teuthology.orchestra.run.smithi042.stdout:749 writesame 0x69c6f52 thru 0x69ced16 data_size 0x1(0x7dc5 bytes) 2024-09-18T00:57:54.932 INFO:teuthology.orchestra.run.smithi042.stdout:750 trunc from 0x8a42fc8 to 0x59dcfd0 2024-09-18T00:57:54.933 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:54.930+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:54.946 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:54.949 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:54.949 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:54.952 INFO:teuthology.orchestra.run.smithi042.stdout:753 trunc from 0x59dcfd0 to 0x16ee3ef 2024-09-18T00:57:54.971 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:57:54.984 INFO:teuthology.orchestra.run.smithi042.stdout:754 trunc from 0x16ee3ef to 0x89c28f 2024-09-18T00:57:55.006 INFO:teuthology.orchestra.run.smithi042.stdout:756 read 0x41415d thru 0x41ab84 (0x6a28 bytes) 2024-09-18T00:57:55.031 INFO:teuthology.orchestra.run.smithi042.stdout:757 write 0xba400c2 thru 0xba4fa2a (0xf969 bytes) 2024-09-18T00:57:55.034 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:55.034 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:55.035 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:55.030+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:55.045 INFO:teuthology.orchestra.run.smithi042.stdout:760 write 0xcb636d9 thru 0xcb6b93f (0x8267 bytes) 2024-09-18T00:57:55.062 INFO:teuthology.orchestra.run.smithi042.stdout:763 writesame 0xd5e24ab thru 0xd5eb67b data_size 0x17(0x91d1 bytes) 2024-09-18T00:57:55.062 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:55.062 INFO:teuthology.orchestra.run.smithi042.stdout:765 punch from 0x26e6e03 to 0x26f347c, (0xc679 bytes) 2024-09-18T00:57:55.062 INFO:teuthology.orchestra.run.smithi042.stdout:766 punch from 0xca5bd0f to 0xca5ffce, (0x42bf bytes) 2024-09-18T00:57:55.062 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:55.062 INFO:teuthology.orchestra.run.smithi042.stdout:770 write 0x6719d4b thru 0x671d7b2 (0x3a68 bytes) 2024-09-18T00:57:55.067 INFO:teuthology.orchestra.run.smithi042.stdout:772 punch from 0xab68170 to 0xab726d5, (0xa565 bytes) 2024-09-18T00:57:55.067 INFO:teuthology.orchestra.run.smithi042.stdout:773 writesame 0xc3bda38 thru 0xc3cc79b data_size 0x2(0xed64 bytes) 2024-09-18T00:57:55.101 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:55.101 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:55.106 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:55.106 INFO:teuthology.orchestra.run.smithi042.stdout:776 write 0xeb0ab5 thru 0xeb86f1 (0x7c3d bytes) 2024-09-18T00:57:55.113 INFO:teuthology.orchestra.run.smithi042.stdout:777 writesame 0x4482289 thru 0x4488024 data_size 0x6(0x5d9c bytes) 2024-09-18T00:57:55.116 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:57:55.131 INFO:teuthology.orchestra.run.smithi042.stdout:778 trunc from 0xd5eb67c to 0x9f40de0 2024-09-18T00:57:55.155 INFO:teuthology.orchestra.run.smithi042.stdout:779 read 0x4458d16 thru 0x4466d0f (0xdffa bytes) 2024-09-18T00:57:55.155 INFO:teuthology.orchestra.run.smithi042.stdout:782 read 0x60e1ddb thru 0x60e5f35 (0x415b bytes) 2024-09-18T00:57:55.155 INFO:teuthology.orchestra.run.smithi042.stdout:783 write 0x3a3afab thru 0x3a3fc5a (0x4cb0 bytes) 2024-09-18T00:57:55.157 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:55.154+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:784 trunc from 0x9f40de0 to 0x4e78b43 2024-09-18T00:57:55.198 INFO:teuthology.orchestra.run.smithi042.stdout:785 read 0x4df8c78 thru 0x4dfa5a4 (0x192d bytes) 2024-09-18T00:57:55.200 INFO:teuthology.orchestra.run.smithi042.stdout:786 trunc from 0x4e78b43 to 0x4716ee2 2024-09-18T00:57:55.215 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xeac7700 2024-09-18T00:57:55.236 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:55.230+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:55.243 INFO:teuthology.orchestra.run.smithi042.stdout:787 trunc from 0x4716ee2 to 0xeac7700 2024-09-18T00:57:55.256 INFO:teuthology.orchestra.run.smithi042.stdout:788 punch from 0xa5fb7fb to 0xa600044, (0x4849 bytes) 2024-09-18T00:57:55.256 INFO:teuthology.orchestra.run.smithi042.stdout:789 read 0x25fac88 thru 0x25fcd0e (0x2087 bytes) 2024-09-18T00:57:55.256 INFO:teuthology.orchestra.run.smithi042.stdout:792 write 0xe90ce32 thru 0xe917f0d (0xb0dc bytes) 2024-09-18T00:57:55.265 INFO:teuthology.orchestra.run.smithi042.stdout:793 writesame 0xa16199a thru 0xa1689b4 data_size 0x1(0x701b bytes) 2024-09-18T00:57:55.278 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:55.278 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:55.293 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:57:55.305 INFO:teuthology.orchestra.run.smithi042.stdout:794 clone 8 order 19 su 262144 sc 10 2024-09-18T00:57:55.338 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:55.334+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:55.438 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:55.434+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:55.460 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:55.461 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:55.474 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:57:55.562 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:55.558+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:55.640 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:55.640 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:55.640 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:55.634+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:55.654 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:57:55.745 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:55.742+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:55.821 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:55.821 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:55.835 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:57:55.878 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:55.874+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:55.942 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:55.938+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:55.999 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:55.999 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:56.014 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:57:56.017 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:57:56.058 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:56.054+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:56.109 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone7 from 0xeac7700 (overlap 0x89c28f) to 0x0 2024-09-18T00:57:56.144 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:56.138+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:56.178 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:56.179 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:56.186 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-18T00:57:56.192 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:57:56.246 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:56.242+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:56.311 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:56.311 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:56.325 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:57:56.347 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:56.342+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:56.446 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:56.447 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:56.448 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:56.442+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:56.461 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:57:56.532 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:57:56.578 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:56.574+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:56.591 INFO:teuthology.orchestra.run.smithi042.stdout:796 read 0x9493443 thru 0x9497806 (0x43c4 bytes) 2024-09-18T00:57:56.598 INFO:teuthology.orchestra.run.smithi042.stdout:798 read 0xc07195e thru 0xc0792fa (0x799d bytes) 2024-09-18T00:57:56.598 INFO:teuthology.orchestra.run.smithi042.stdout:799 write 0xc0f8de6 thru 0xc1083d0 (0xf5eb bytes) 2024-09-18T00:57:56.610 INFO:teuthology.orchestra.run.smithi042.stdout:800 read 0x588c3df thru 0x588dadd (0x16ff bytes) 2024-09-18T00:57:56.611 INFO:teuthology.orchestra.run.smithi042.stdout:801 write 0x77d804f thru 0x77e1194 (0x9146 bytes) 2024-09-18T00:57:56.618 INFO:teuthology.orchestra.run.smithi042.stdout:802 writesame 0xd7616c4 thru 0xd761813 data_size 0x8(0x150 bytes) 2024-09-18T00:57:56.635 INFO:teuthology.orchestra.run.smithi042.stdout:803 trunc from 0xeac7700 to 0x7a02d7f 2024-09-18T00:57:56.636 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:56.637 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:56.650 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:56.646+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:56.654 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:57:56.658 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:57:56.659 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:57:56.659 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-18T00:57:56.659 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T00:57:56.659 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T00:57:56.659 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-18T00:57:56.674 INFO:teuthology.orchestra.run.smithi042.stdout:804 read 0x5cb3d6f thru 0x5cbf962 (0xbbf4 bytes) 2024-09-18T00:57:56.674 INFO:teuthology.orchestra.run.smithi042.stdout:805 read 0x347c68e thru 0x34821ab (0x5b1e bytes) 2024-09-18T00:57:56.689 INFO:teuthology.orchestra.run.smithi042.stdout:807 writesame 0xd0a8e06 thru 0xd0ae01d data_size 0x8(0x5218 bytes) 2024-09-18T00:57:56.689 INFO:teuthology.orchestra.run.smithi042.stdout:809 read 0x5a7511d thru 0x5a809fd (0xb8e1 bytes) 2024-09-18T00:57:56.689 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:57:56.690 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:57:56.693 INFO:teuthology.orchestra.run.smithi042.stdout:810 trunc from 0xd0ae01e to 0xa3a7e88 2024-09-18T00:57:56.708 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:57:56.709 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:57:56.710 INFO:teuthology.orchestra.run.smithi042.stdout:811 write 0xd289113 thru 0xd29592c (0xc81a bytes) 2024-09-18T00:57:56.714 INFO:teuthology.orchestra.run.smithi042.stdout:812 write 0x42a2660 thru 0x42a6702 (0x40a3 bytes) 2024-09-18T00:57:56.718 INFO:teuthology.orchestra.run.smithi042.stdout:814 read 0xaa5119f thru 0xaa522ed (0x114f bytes) 2024-09-18T00:57:56.718 INFO:teuthology.orchestra.run.smithi042.stdout:815 read 0x9adaa8c thru 0x9ae57d7 (0xad4c bytes) 2024-09-18T00:57:56.718 INFO:teuthology.orchestra.run.smithi042.stdout:816 read 0x3835323 thru 0x38382a9 (0x2f87 bytes) 2024-09-18T00:57:56.719 INFO:teuthology.orchestra.run.smithi042.stdout:817 writesame 0x8a3d9b4 thru 0x8a4137c data_size 0x1(0x39c9 bytes) 2024-09-18T00:57:56.726 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:57:56.726 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:57:56.729 INFO:teuthology.orchestra.run.smithi042.stdout:818 punch from 0x15a68bd to 0x15b12eb, (0xaa2e bytes) 2024-09-18T00:57:56.729 INFO:teuthology.orchestra.run.smithi042.stdout:819 write 0x10cb0c3 thru 0x10d0576 (0x54b4 bytes) 2024-09-18T00:57:56.733 INFO:teuthology.orchestra.run.smithi042.stdout:820 read 0x24e62f4 thru 0x24f292d (0xc63a bytes) 2024-09-18T00:57:56.733 INFO:teuthology.orchestra.run.smithi042.stdout:821 punch from 0x571eb95 to 0x572af9f, (0xc40a bytes) 2024-09-18T00:57:56.733 INFO:teuthology.orchestra.run.smithi042.stdout:823 write 0xb402db1 thru 0xb404a61 (0x1cb1 bytes) 2024-09-18T00:57:56.737 INFO:teuthology.orchestra.run.smithi042.stdout:824 read 0x130ede2 thru 0x1318ddf (0x9ffe bytes) 2024-09-18T00:57:56.737 INFO:teuthology.orchestra.run.smithi042.stdout:825 write 0xf29f6b thru 0xf3123e (0x72d4 bytes) 2024-09-18T00:57:56.741 INFO:teuthology.orchestra.run.smithi042.stdout:826 write 0x1caf6f8 thru 0x1cbc30c (0xcc15 bytes) 2024-09-18T00:57:56.744 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:57:56.744 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:57:56.745 INFO:teuthology.orchestra.run.smithi042.stdout:827 writesame 0xb465c8a thru 0xb46f556 data_size 0x27(0x98cd bytes) 2024-09-18T00:57:56.749 INFO:teuthology.orchestra.run.smithi042.stdout:828 punch from 0x9753fbd to 0x975b02f, (0x7072 bytes) 2024-09-18T00:57:56.749 INFO:teuthology.orchestra.run.smithi042.stdout:829 write 0xae049e thru 0xae7239 (0x6d9c bytes) 2024-09-18T00:57:56.750 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:56.746+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:56.753 INFO:teuthology.orchestra.run.smithi042.stdout:830 write 0xaeaaab thru 0xaf18af (0x6e05 bytes) 2024-09-18T00:57:56.755 INFO:teuthology.orchestra.run.smithi042.stdout:831 read 0x678874f thru 0x678d7c8 (0x507a bytes) 2024-09-18T00:57:56.764 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:57:56.765 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:57:56.769 INFO:teuthology.orchestra.run.smithi042.stdout:833 trunc from 0xd29592d to 0x388dd81 2024-09-18T00:57:56.787 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:57:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:57:56.798 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:56.798 INFO:teuthology.orchestra.run.smithi042.stdout:837 read 0x304dd44 thru 0x305547f (0x773c bytes) 2024-09-18T00:57:56.798 INFO:teuthology.orchestra.run.smithi042.stdout:839 writesame 0x56e6f0 thru 0x572025 data_size 0x3(0x3936 bytes) 2024-09-18T00:57:56.801 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:56.802 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:56.805 INFO:teuthology.orchestra.run.smithi042.stdout:840 write 0x2eac9ab thru 0x2eaca6d (0xc3 bytes) 2024-09-18T00:57:56.809 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:57:56.810 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:57:56.810 INFO:teuthology.orchestra.run.smithi042.stdout:841 read 0x14935da thru 0x14a14a3 (0xdeca bytes) 2024-09-18T00:57:56.811 INFO:teuthology.orchestra.run.smithi042.stdout:842 flatten 2024-09-18T00:57:56.820 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:57:56.835 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:57:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:57:56.841 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T00:57:56.841 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:56.851 INFO:teuthology.orchestra.run.smithi042.stdout:843 write 0xae13991 thru 0xae17e8f (0x44ff bytes) 2024-09-18T00:57:56.855 INFO:teuthology.orchestra.run.smithi042.stdout:844 punch from 0x21374a1 to 0x213f070, (0x7bcf bytes) 2024-09-18T00:57:56.856 INFO:teuthology.orchestra.run.smithi042.stdout:847 write 0x68a453b thru 0x68a8b6f (0x4635 bytes) 2024-09-18T00:57:56.857 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:56.850+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:56.859 INFO:teuthology.orchestra.run.smithi042.stdout:848 read 0x710c63b thru 0x710dc74 (0x163a bytes) 2024-09-18T00:57:56.859 INFO:teuthology.orchestra.run.smithi042.stdout:849 write 0x10b86ec thru 0x10bb59d (0x2eb2 bytes) 2024-09-18T00:57:56.870 INFO:teuthology.orchestra.run.smithi042.stdout:851 write 0xd4fd604 thru 0xd502e6e (0x586b bytes) 2024-09-18T00:57:56.874 INFO:teuthology.orchestra.run.smithi042.stdout:852 write 0x1fe1aad thru 0x1fe34bc (0x1a10 bytes) 2024-09-18T00:57:56.877 INFO:teuthology.orchestra.run.smithi042.stdout:853 write 0x18eba2a thru 0x18f9834 (0xde0b bytes) 2024-09-18T00:57:56.881 INFO:teuthology.orchestra.run.smithi042.stdout:854 write 0x44c451e thru 0x44c6597 (0x207a bytes) 2024-09-18T00:57:56.884 INFO:teuthology.orchestra.run.smithi042.stdout:856 read 0x7030579 thru 0x703dcc9 (0xd751 bytes) 2024-09-18T00:57:56.885 INFO:teuthology.orchestra.run.smithi042.stdout:857 writesame 0x4c54959 thru 0x4c5d6a4 data_size 0x2(0x8d4c bytes) 2024-09-18T00:57:56.902 INFO:teuthology.orchestra.run.smithi042.stdout:858 trunc from 0xd502e6f to 0xb7cb15d 2024-09-18T00:57:56.932 INFO:teuthology.orchestra.run.smithi042.stdout:859 trunc from 0xb7cb15d to 0x56856b 2024-09-18T00:57:56.951 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:56.946+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:56.966 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:57:56.966 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:57:56.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:57:56.966 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:57:56.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.966 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4041 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:56.966 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:57:56.966 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.961449084, 2024-09-18T00:57:56.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037088000000000003, 2024-09-18T00:57:56.966 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4041 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003005+0000", 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.187e-06 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003011+0000", 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003029+0000", 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8026999999999999e-05 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003043+0000", 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039484+0000", 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036440895000000001 2024-09-18T00:57:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039722+0000", 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000238507 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040092+0000", 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000369818 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.961391461, 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.961333204000001, 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.961274765000001, 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:57:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:56.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.961213548, 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.984 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:57:56.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:57:56.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:57:56.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.961123132999999, 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:56.988 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:56.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.992 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.961052199999999, 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:56.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:57:56.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:56.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:57:56.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.960992594, 2024-09-18T00:57:56.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:56.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:57:57.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.960933224, 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.002 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:57:57.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.006 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.960871435, 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.007 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:57:57.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.960738612, 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:57:57.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:57:57.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4055211350000008, 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T00:57:57.016 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T00:57:57.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T00:57:57.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8459 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559110+0000", 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4053435590000003, 2024-09-18T00:57:57.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037081250000000003, 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8459 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559110+0000", 2024-09-18T00:57:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559110+0000", 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559110+0000", 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2099999999999996e-07 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559123+0000", 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2809000000000001e-05 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559130+0000", 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2339999999999997e-06 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559364+0000", 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023426299999999999 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596092+0000", 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036727965000000001 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596146+0000", 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3372000000000003e-05 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596191+0000", 2024-09-18T00:57:57.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4985999999999999e-05 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4050210110000005, 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.027 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:57:57.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.031 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4049905030000005, 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:57:57.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T00:57:57.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:57:57.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T00:57:57.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T00:57:57.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:57.036 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4049658380000007, 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:57:57.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:57:57.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T00:57:57.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T00:57:57.041 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.041 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.041 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.041 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:57.041 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:57:57.041 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4049373060000008, 2024-09-18T00:57:57.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T00:57:57.041 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.041 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T00:57:57.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T00:57:57.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T00:57:57.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T00:57:57.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T00:57:57.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T00:57:57.045 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4046040150000003, 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.046 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T00:57:57.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T00:57:57.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4045621639999997, 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.050 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T00:57:57.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T00:57:57.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4035246679999993, 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T00:57:57.056 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T00:57:57.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:57:57.060 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:57:57.062 INFO:teuthology.orchestra.run.smithi042.stdout:861 writesame 0x7286d96 thru 0x72903d1 data_size 0x1e(0x963c bytes) 2024-09-18T00:57:57.062 INFO:teuthology.orchestra.run.smithi042.stdout:862 punch from 0x4cf5c2e to 0x4cfb551, (0x5923 bytes) 2024-09-18T00:57:57.062 INFO:teuthology.orchestra.run.smithi042.stdout:863 writesame 0xd1433d thru 0xd23f41 data_size 0x1(0xfc05 bytes) 2024-09-18T00:57:57.063 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:57:57.058+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:57:57.063 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:57:57.063 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:57:57.064 INFO:teuthology.orchestra.run.smithi042.stdout:864 trunc from 0x72903d2 to 0x627dd61 2024-09-18T00:57:57.064 INFO:teuthology.orchestra.run.smithi042.stdout:865 trunc from 0x627dd61 to 0x4eec043 2024-09-18T00:57:57.065 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:57.065 INFO:teuthology.orchestra.run.smithi042.stdout:867 trunc from 0x4eec043 to 0x4caccf2 2024-09-18T00:57:57.090 INFO:teuthology.orchestra.run.smithi042.stdout:868 write 0xa34005e thru 0xa3465ff (0x65a2 bytes) 2024-09-18T00:57:57.098 INFO:teuthology.orchestra.run.smithi042.stdout:870 writesame 0x3eb274c thru 0x3ec180a data_size 0x1(0xf0bf bytes) 2024-09-18T00:57:57.163 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:57.158+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:57.175 INFO:teuthology.orchestra.run.smithi042.stdout:871 trunc from 0xa346600 to 0x721fd50 2024-09-18T00:57:57.198 INFO:teuthology.orchestra.run.smithi042.stdout:872 write 0x41dcd30 thru 0x41dfda8 (0x3079 bytes) 2024-09-18T00:57:57.205 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:57.205 INFO:teuthology.orchestra.run.smithi042.stdout:876 clone 9 order 22 su 16384 sc 7 2024-09-18T00:57:57.221 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:57.222 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:57.222 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:57.236 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:57:57.278 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:57.274+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:57.365 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:57.358+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:57.396 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:57.396 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:57.396 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:57.396 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:57.396 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:57.411 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:57:57.498 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:57.494+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:57.568 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:57.562+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:57.578 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:57.578 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:57:57.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3710 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.707111572000001, 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024498702000000001, 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3710 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867971+0000", 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.671e-06 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867976+0000", 2024-09-18T00:57:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7030000000000003e-06 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867981+0000", 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8520000000000003e-06 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892175+0000", 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024193538000000001 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892279+0000", 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104017 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892467+0000", 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018858699999999999 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3711 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.707093252, 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024701471999999999, 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3711 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6199999999999999e-07 2024-09-18T00:57:57.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867989+0000", 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867995+0000", 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.127e-06 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868000+0000", 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-18T00:57:57.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892500+0000", 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024499725999999999 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892587+0000", 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7232999999999996e-05 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892688+0000", 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010115000000000001 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3712 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.707074294, 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024914385000000001, 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3712 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868008+0000", 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868014+0000", 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868019+0000", 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892727+0000", 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024707638000000001 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892817+0000", 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9569999999999998e-05 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892920+0000", 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103744 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3713 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.707054763, 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025115858000000001, 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3713 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.593 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868026+0000", 2024-09-18T00:57:57.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.58e-07 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868032+0000", 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3790000000000003e-06 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868039+0000", 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6969999999999996e-06 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868044+0000", 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7060000000000003e-06 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892954+0000", 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024910704999999998 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893041+0000", 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6747000000000001e-05 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893141+0000", 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010026600000000001 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3714 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.707029956, 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025291557999999999, 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3714 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868051+0000", 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868052+0000", 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.863e-06 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868059+0000", 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2720000000000003e-06 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868064+0000", 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8860000000000003e-06 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893175+0000", 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025111114 2024-09-18T00:57:57.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893262+0000", 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7476000000000001e-05 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893342+0000", 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9577000000000001e-05 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3715 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.707010451, 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025471191000000001, 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3715 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2799999999999996e-07 2024-09-18T00:57:57.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868072+0000", 2024-09-18T00:57:57.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9269999999999999e-06 2024-09-18T00:57:57.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868078+0000", 2024-09-18T00:57:57.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9830000000000001e-06 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868083+0000", 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.972e-06 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893373+0000", 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025290316 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893463+0000", 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9078000000000003e-05 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893541+0000", 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8386999999999993e-05 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3716 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.606 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:57.607 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.706991392999999, 2024-09-18T00:57:57.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025664781000000001, 2024-09-18T00:57:57.607 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3716 2024-09-18T00:57:57.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2099999999999998e-07 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868091+0000", 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8950000000000001e-06 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868097+0000", 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1929999999999998e-06 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868102+0000", 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6789999999999996e-06 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893570+0000", 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025468227999999999 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893657+0000", 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7176999999999996e-05 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893754+0000", 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6287999999999997e-05 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3717 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.611 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.706972286999999, 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025815932999999999, 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3717 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868111+0000", 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6860000000000002e-06 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868117+0000", 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868122+0000", 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.69e-06 2024-09-18T00:57:57.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893802+0000", 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025680397000000001 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893869+0000", 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6597e-05 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893924+0000", 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5031999999999998e-05 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3718 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.706951265000001, 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026018052, 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3718 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:57.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868131+0000", 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9369999999999998e-06 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868138+0000", 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4389999999999997e-06 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868143+0000", 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1440000000000002e-06 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893973+0000", 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025830490000000001 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894063+0000", 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9313000000000004e-05 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894147+0000", 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4342e-05 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.620 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3719 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.706931309, 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026147934000000001, 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3719 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-18T00:57:57.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868151+0000", 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868157+0000", 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868162+0000", 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5619999999999997e-06 2024-09-18T00:57:57.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894183+0000", 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026021083 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894241+0000", 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8505999999999999e-05 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894297+0000", 2024-09-18T00:57:57.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5396000000000003e-05 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3720 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.706912792000001, 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026376077000000001, 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3720 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:57:57.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868168+0000", 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868170+0000", 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0889999999999998e-06 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868176+0000", 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3090000000000001e-06 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868181+0000", 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8810000000000004e-06 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894341+0000", 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026159636 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894430+0000", 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8986000000000005e-05 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894543+0000", 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011382700000000001 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.629 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3721 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.706893274, 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569722, 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3721 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868189+0000", 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7460000000000001e-06 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868195+0000", 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8490000000000002e-06 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868200+0000", 2024-09-18T00:57:57.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1669999999999998e-06 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894623+0000", 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02642338 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894688+0000", 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4183000000000005e-05 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894757+0000", 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9062999999999999e-05 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3722 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.706874589, 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026735041000000001, 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3722 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868208+0000", 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8470000000000001e-06 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868214+0000", 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1090000000000003e-06 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868218+0000", 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.476e-06 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894788+0000", 2024-09-18T00:57:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569274 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894868+0000", 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0711999999999996e-05 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894941+0000", 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3723 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.706856173, 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026971922999999998, 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3723 2024-09-18T00:57:57.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-07 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868226+0000", 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.812e-06 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868232+0000", 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7910000000000001e-06 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868237+0000", 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7339999999999999e-06 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895056+0000", 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026819195000000001 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895103+0000", 2024-09-18T00:57:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7264e-05 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895196+0000", 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2802999999999999e-05 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3724 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.706837961, 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027181833999999998, 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3724 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868243+0000", 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868245+0000", 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1160000000000002e-06 2024-09-18T00:57:57.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868251+0000", 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1410000000000002e-06 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868256+0000", 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0130000000000001e-06 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895249+0000", 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02699352 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895338+0000", 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8331000000000005e-05 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895424+0000", 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6348999999999996e-05 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3725 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.706818826999999, 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027296051000000002, 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3725 2024-09-18T00:57:57.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868262+0000", 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868267+0000", 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868274+0000", 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2360000000000001e-06 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868278+0000", 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7620000000000003e-06 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895458+0000", 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027179608000000001 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895511+0000", 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2738000000000002e-05 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895557+0000", 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6631999999999997e-05 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3727 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.703508276000001, 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024398075000000002, 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3727 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871573+0000", 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.367e-06 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871580+0000", 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8029999999999999e-06 2024-09-18T00:57:57.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871600+0000", 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0213000000000001e-05 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871615+0000", 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.471e-05 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895805+0000", 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024189677 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895923+0000", 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118496 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895970+0000", 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6808999999999999e-05 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3728 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.703447017, 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024637071999999999, 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3728 2024-09-18T00:57:57.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871634+0000", 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.074e-06 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871642+0000", 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8830000000000006e-06 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871660+0000", 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8283000000000001e-05 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871692+0000", 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1714000000000002e-05 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896004+0000", 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024312099 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896099+0000", 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5036000000000003e-05 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896270+0000", 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017098300000000001 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.5111151579999991, 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:57:57.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:57:57.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:57:57.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15723 3.f 3:f7b50ba2:::rbd_data.117c53e529c1.0000000000000058:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:57:57.667 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.3528462320000001, 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026056645999999999, 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15723 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222235+0000", 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222242+0000", 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4710000000000004e-06 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222255+0000", 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2877999999999999e-05 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222261+0000", 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.489e-06 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222328+0000", 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7280999999999995e-05 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222502+0000", 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000173609 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.227120+0000", 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046180129999999998 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:57.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248104+0000", 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020983979 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248166+0000", 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2340000000000003e-05 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248236+0000", 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9227000000000007e-05 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248291+0000", 2024-09-18T00:57:57.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5019000000000003e-05 2024-09-18T00:57:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:57.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:57.673 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:57.674 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:57:57.675 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:57.670+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:57.776 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:57.770+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:57.849 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:57.850 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:57.850 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:57.850 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:57.864 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:57:57.876 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:57.870+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:57.977 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:57.970+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:58.049 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:58.049 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:58.049 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:58.049 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:58.049 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:58.066 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:57:58.078 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:58.074+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:58.178 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:57:58.174+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:57:58.223 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-18T00:57:58.234 INFO:teuthology.orchestra.run.smithi042.stdout:878 trunc from 0x721fd50 to 0x75c1170 2024-09-18T00:57:58.245 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:57:58.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.953239570000001, 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:57:58.246 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:57:58.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:57:58.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:57:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:57:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:57:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:57:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:57:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:57:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:57:58.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7423 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7569410199999993, 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045296661000000002, 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7423 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485621+0000", 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485627+0000", 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.156e-06 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485633+0000", 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789999999999999e-06 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530788+0000", 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045155001 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530853+0000", 2024-09-18T00:57:58.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5634000000000002e-05 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530915+0000", 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2022999999999999e-05 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7424 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1146880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7569200299999999, 2024-09-18T00:57:58.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045419355000000002, 2024-09-18T00:57:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7424 2024-09-18T00:57:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:57:58.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485642+0000", 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0640000000000001e-06 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485649+0000", 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485654+0000", 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2329999999999997e-06 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530941+0000", 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045286602000000002 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531004+0000", 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3455999999999997e-05 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531059+0000", 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4771000000000001e-05 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7568985169999998, 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:57:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:57:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:57:58.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7568770800000006, 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:57:58.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:57:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:57:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:57:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:57:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.271 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7567817530000003, 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.272 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:57:58.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:57:58.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:57:58.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:57:58.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.276 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.277 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.277 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:58.277 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7567563140000004, 2024-09-18T00:57:58.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:57:58.277 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:57:58.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:57:58.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:57:58.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:57:58.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:57:58.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.281 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.281 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7567337910000003, 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:58.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:57:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:57:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7567103510000006, 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.286 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:57:58.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7566865410000005, 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:57:58.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7566641759999992, 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.298 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7566375589999996, 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.299 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:57:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:57:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:57:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:57:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:57:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:57:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:57:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7524857879999995, 2024-09-18T00:57:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:57:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:57:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7524650449999992, 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:57:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:57:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:57:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7524194800000004, 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:57:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:57:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7523931390000005, 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:57:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7523704690000006, 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:57:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:57:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7523520959999992, 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7523323079999997, 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:57:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:57:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7523054380000005, 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:57:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:57:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:57:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.339 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:58.340 INFO:teuthology.orchestra.run.smithi042.stdout:879 read 0x2c77575 thru 0x2c7e34b (0x6dd7 bytes) 2024-09-18T00:57:58.340 INFO:teuthology.orchestra.run.smithi042.stdout:880 writesame 0x4e9f630 thru 0x4ea0341 data_size 0x7(0xd12 bytes) 2024-09-18T00:57:58.341 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:57:58.341 INFO:teuthology.orchestra.run.smithi042.stdout:883 write 0xd429ff4 thru 0xd43172a (0x7737 bytes) 2024-09-18T00:57:58.342 INFO:teuthology.orchestra.run.smithi042.stdout:884 read 0x4ca3221 thru 0x4ca3636 (0x416 bytes) 2024-09-18T00:57:58.342 INFO:teuthology.orchestra.run.smithi042.stdout:885 trunc from 0xd43172b to 0x5d1383e 2024-09-18T00:57:58.342 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:57:58.334+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:57:58.354 INFO:teuthology.orchestra.run.smithi042.stdout:886 trunc from 0x5d1383e to 0xe474890 2024-09-18T00:57:58.362 INFO:teuthology.orchestra.run.smithi042.stdout:887 read 0x881f236 thru 0x882e95a (0xf725 bytes) 2024-09-18T00:57:58.362 INFO:teuthology.orchestra.run.smithi042.stdout:888 clone 10 order 24 su 262144 sc 5 2024-09-18T00:57:58.441 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:58.434+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:58.477 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:58.477 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:58.477 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:58.477 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:58.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:57:58.542 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:58.538+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:58.643 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:58.638+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:58.657 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:58.657 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:58.657 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:57:58.658 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:57:58.658 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:58.672 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:57:58.744 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:58.738+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:58.797 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:58.797 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:57:58.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:57:58.797 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:57:58.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.797 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.797 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:57:58.797 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.716146179000001, 2024-09-18T00:57:58.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:57:58.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:57:58.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.715216588000001, 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:57:58.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:57:58.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:57:58.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:57:58.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:57:58.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.807 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.807 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.807 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:57:58.807 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.715202528000001, 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:57:58.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:57:58.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:57:58.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.812 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.812 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.812 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:57:58.812 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.713911953, 2024-09-18T00:57:58.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:57:58.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:57:58.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:57:58.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:57:58.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.713891480999999, 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:57:58.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.711097027999999, 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:57:58.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.711081844000001, 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:57:58.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:57:58.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.831 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.711058897999999, 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:57:58.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:57:58.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1096 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.706896559, 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058813730000000002, 2024-09-18T00:57:58.836 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1096 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.75e-07 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086206+0000", 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.677e-06 2024-09-18T00:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086210+0000", 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5950000000000001e-06 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086214+0000", 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4979999999999998e-06 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.144798+0000", 2024-09-18T00:57:58.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058584587 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.144893+0000", 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4413000000000006e-05 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145017+0000", 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000124485 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.706881258999999, 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:57:58.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:57:58.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:57:58.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:57:58.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:57:58.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:57:58.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:57:58.845 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.706860042000001, 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:57:58.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.706838659000001, 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:57:58.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:57:58.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.702938805, 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:57:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:57:58.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.702924604, 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.702906566999999, 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.702892501000001, 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.69886715, 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.698853138, 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:57:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.698839174, 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:57:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9369812199999998, 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:57:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:57:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:57:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:57:58.894 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:58.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:57:58.895 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:58.890+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:58.896 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:57:58.996 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:58.990+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:59.065 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:57:59.071 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:57:59.071 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:57:59.071 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:57:59.071 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:57:59.092 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:57:59.097 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:57:59.090+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:57:59.110 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone9 from 0xe474890 (overlap 0x5d1383e) to 0x1bec412 2024-09-18T00:57:59.112 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:57:59.112 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:57:59.148 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:57:59.148 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:57:59.183 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:57:59.183 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:57:59.197 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:59.190+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:57:59.217 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-18T00:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:57:59.265 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:57:59.286 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:57:59.298 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:59.298 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:59.298 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:59.294+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:59.307 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:57:59.307 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:57:59.323 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:57:59.426 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:57:59.422+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:57:59.471 INFO:teuthology.orchestra.run.smithi042.stdout:889 write 0xc9a0daf thru 0xc9a44a4 (0x36f6 bytes) 2024-09-18T00:57:59.485 INFO:teuthology.orchestra.run.smithi042.stdout:891 read 0x7f5e4f7 thru 0x7f64834 (0x633e bytes) 2024-09-18T00:57:59.487 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:59.487 INFO:teuthology.orchestra.run.smithi042.stdout:893 read 0x2205c6 thru 0x226038 (0x5a73 bytes) 2024-09-18T00:57:59.495 INFO:teuthology.orchestra.run.smithi042.stdout:894 punch from 0xd0fe1d4 to 0xd101e7c, (0x3ca8 bytes) 2024-09-18T00:57:59.501 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:59.494+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:59.509 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:59.509 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:59.516 INFO:teuthology.orchestra.run.smithi042.stdout:895 trunc from 0xe474890 to 0x16d823 2024-09-18T00:57:59.530 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:57:59.609 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:57:59.598+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:57:59.651 INFO:teuthology.orchestra.run.smithi042.stdout:896 writesame 0xb83ab48 thru 0xb842c57 data_size 0x76(0x8110 bytes) 2024-09-18T00:57:59.651 INFO:teuthology.orchestra.run.smithi042.stdout:899 punch from 0x9cfba7 to 0x9d0065, (0x4be bytes) 2024-09-18T00:57:59.651 INFO:teuthology.orchestra.run.smithi042.stdout:900 write 0x8cdd79f thru 0x8cec5f3 (0xee55 bytes) 2024-09-18T00:57:59.655 INFO:teuthology.orchestra.run.smithi042.stdout:901 write 0x65c8785 thru 0x65cdc15 (0x5491 bytes) 2024-09-18T00:57:59.658 INFO:teuthology.orchestra.run.smithi042.stdout:903 read 0x99116e4 thru 0x9913b7e (0x249b bytes) 2024-09-18T00:57:59.659 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:59.659 INFO:teuthology.orchestra.run.smithi042.stdout:905 read 0x293ead6 thru 0x2946a41 (0x7f6c bytes) 2024-09-18T00:57:59.659 INFO:teuthology.orchestra.run.smithi042.stdout:906 punch from 0x17d5e33 to 0x17d93c5, (0x3592 bytes) 2024-09-18T00:57:59.659 INFO:teuthology.orchestra.run.smithi042.stdout:908 write 0x231e80c thru 0x232735f (0x8b54 bytes) 2024-09-18T00:57:59.661 INFO:teuthology.orchestra.run.smithi042.stdout:909 punch from 0x5b52924 to 0x5b56020, (0x36fc bytes) 2024-09-18T00:57:59.661 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:57:59.661 INFO:teuthology.orchestra.run.smithi042.stdout:913 write 0xa813b02 thru 0xa816d7c (0x327b bytes) 2024-09-18T00:57:59.665 INFO:teuthology.orchestra.run.smithi042.stdout:914 clone 11 order 18 su 16384 sc 2 2024-09-18T00:57:59.682 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:59.682 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:59.696 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:57:59.703 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:57:59.698+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:57:59.803 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:59.798+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:59.857 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:59.857 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:57:59.872 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:57:59.904 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:57:59.902+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:57:59.997 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:57:59.997 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:00.005 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:00.002+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:00.011 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:58:00.106 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:00.102+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:00.128 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone10 from 0xb842c58 (overlap 0x16d823) to 0xdb4e1 2024-09-18T00:58:00.187 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:00.187 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:00.201 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:00.207 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:00.202+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:00.272 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-18T00:58:00.307 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:00.302+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:00.325 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:00.325 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:00.342 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:00.437 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:00.434+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:00.452 INFO:teuthology.orchestra.run.smithi042.stdout:917 write 0xe5cda42 thru 0xe5dca71 (0xf030 bytes) 2024-09-18T00:58:00.460 INFO:teuthology.orchestra.run.smithi042.stdout:918 write 0x2d8aa24 thru 0x2d8f3ea (0x49c7 bytes) 2024-09-18T00:58:00.475 INFO:teuthology.orchestra.run.smithi042.stdout:919 punch from 0x4d3b524 to 0x4d3ca9c, (0x1578 bytes) 2024-09-18T00:58:00.476 INFO:teuthology.orchestra.run.smithi042.stdout:920 read 0x33f719 thru 0x342887 (0x316f bytes) 2024-09-18T00:58:00.477 INFO:teuthology.orchestra.run.smithi042.stdout:921 write 0x40074ed thru 0x4010677 (0x918b bytes) 2024-09-18T00:58:00.509 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:00.502+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:00.512 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:00.512 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:00.521 INFO:teuthology.orchestra.run.smithi042.stdout:924 trunc from 0xe5dca72 to 0x703902b 2024-09-18T00:58:00.526 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:00.556 INFO:teuthology.orchestra.run.smithi042.stdout:925 read 0x1e8c780 thru 0x1e92f03 (0x6784 bytes) 2024-09-18T00:58:00.585 INFO:teuthology.orchestra.run.smithi042.stdout:927 write 0x7db18b4 thru 0x7dbb936 (0xa083 bytes) 2024-09-18T00:58:00.594 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:00.594 INFO:teuthology.orchestra.run.smithi042.stdout:930 read 0x4f924bd thru 0x4f9452c (0x2070 bytes) 2024-09-18T00:58:00.596 INFO:teuthology.orchestra.run.smithi042.stdout:931 write 0xfbb5b0 thru 0xfbeec2 (0x3913 bytes) 2024-09-18T00:58:00.610 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:00.606+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:00.617 INFO:teuthology.orchestra.run.smithi042.stdout:932 read 0x63b41fa thru 0x63c2f01 (0xed08 bytes) 2024-09-18T00:58:00.619 INFO:teuthology.orchestra.run.smithi042.stdout:935 read 0xaca1ad thru 0xad69e1 (0xc835 bytes) 2024-09-18T00:58:00.652 INFO:teuthology.orchestra.run.smithi042.stdout:936 writesame 0xafcbb6d thru 0xafd6433 data_size 0x1(0xa8c7 bytes) 2024-09-18T00:58:00.652 INFO:teuthology.orchestra.run.smithi042.stdout:937 punch from 0x35aef04 to 0x35b7ccd, (0x8dc9 bytes) 2024-09-18T00:58:00.652 INFO:teuthology.orchestra.run.smithi042.stdout:938 read 0x4816dff thru 0x48239bf (0xcbc1 bytes) 2024-09-18T00:58:00.654 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:00.654 INFO:teuthology.orchestra.run.smithi042.stdout:941 write 0x16c2e22 thru 0x16c5751 (0x2930 bytes) 2024-09-18T00:58:00.694 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:00.694 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:00.708 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:00.711 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:00.706+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:00.811 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:00.806+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:00.830 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:00.831 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:00.845 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:00.886 INFO:teuthology.orchestra.run.smithi042.stdout:943 trunc from 0xafd6434 to 0x5c980d8 2024-09-18T00:58:00.919 INFO:teuthology.orchestra.run.smithi042.stdout:944 write 0x5c40f0a thru 0x5c472aa (0x63a1 bytes) 2024-09-18T00:58:00.929 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:00.926+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:00.934 INFO:teuthology.orchestra.run.smithi042.stdout:945 write 0x21c966e thru 0x21d56b8 (0xc04b bytes) 2024-09-18T00:58:00.946 INFO:teuthology.orchestra.run.smithi042.stdout:948 punch from 0x30716d2 to 0x307bc37, (0xa565 bytes) 2024-09-18T00:58:00.946 INFO:teuthology.orchestra.run.smithi042.stdout:949 punch from 0xe3ac5 to 0xea0c4, (0x65ff bytes) 2024-09-18T00:58:00.946 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:00.961 INFO:teuthology.orchestra.run.smithi042.stdout:952 write 0x6f6f0fe thru 0x6f789dd (0x98e0 bytes) 2024-09-18T00:58:00.970 INFO:teuthology.orchestra.run.smithi042.stdout:953 punch from 0xbf9578 to 0xbfccb7, (0x373f bytes) 2024-09-18T00:58:00.970 INFO:teuthology.orchestra.run.smithi042.stdout:954 read 0x3bf86c2 thru 0x3c05ac7 (0xd406 bytes) 2024-09-18T00:58:00.973 INFO:teuthology.orchestra.run.smithi042.stdout:955 write 0x28814c4 thru 0x2888af2 (0x762f bytes) 2024-09-18T00:58:01.005 INFO:teuthology.orchestra.run.smithi042.stdout:957 trunc from 0x6f789de to 0x7a7e37a 2024-09-18T00:58:01.013 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:01.010+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:01.018 INFO:teuthology.orchestra.run.smithi042.stdout:959 punch from 0x5ecbfe4 to 0x5ecc47f, (0x49b bytes) 2024-09-18T00:58:01.018 INFO:teuthology.orchestra.run.smithi042.stdout:960 read 0x2ba9106 thru 0x2bb7e2b (0xed26 bytes) 2024-09-18T00:58:01.020 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:01.020 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:01.037 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:01.045 INFO:teuthology.orchestra.run.smithi042.stdout:962 write 0x8b1f9d6 thru 0x8b2bc96 (0xc2c1 bytes) 2024-09-18T00:58:01.073 INFO:teuthology.orchestra.run.smithi042.stdout:963 trunc from 0x8b2bc97 to 0x1a3773a 2024-09-18T00:58:01.125 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:01.122+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:01.156 INFO:teuthology.orchestra.run.smithi042.stdout:964 writesame 0xc4d8d52 thru 0xc4e7c98 data_size 0x1(0xef47 bytes) 2024-09-18T00:58:01.156 INFO:teuthology.orchestra.run.smithi042.stdout:965 punch from 0x61fac64 to 0x6201b8d, (0x6f29 bytes) 2024-09-18T00:58:01.170 INFO:teuthology.orchestra.run.smithi042.stdout:967 write 0xca2a36d thru 0xca39289 (0xef1d bytes) 2024-09-18T00:58:01.179 INFO:teuthology.orchestra.run.smithi042.stdout:968 write 0x26d2e06 thru 0x26e16a4 (0xe89f bytes) 2024-09-18T00:58:01.189 INFO:teuthology.orchestra.run.smithi042.stdout:969 writesame 0x155069d thru 0x155e667 data_size 0x71(0xdfcb bytes) 2024-09-18T00:58:01.203 INFO:teuthology.orchestra.run.smithi042.stdout:970 writesame 0x93a7898 thru 0x93aa26c data_size 0x1(0x29d5 bytes) 2024-09-18T00:58:01.209 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:01.209 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:01.214 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:01.210+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:01.215 INFO:teuthology.orchestra.run.smithi042.stdout:971 write 0xba55339 thru 0xba5b5d5 (0x629d bytes) 2024-09-18T00:58:01.224 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:01.224 INFO:teuthology.orchestra.run.smithi042.stdout:973 read 0xc031023 thru 0xc033b67 (0x2b45 bytes) 2024-09-18T00:58:01.224 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:01.224 INFO:teuthology.orchestra.run.smithi042.stdout:977 punch from 0x1384b3c to 0x138bc36, (0x70fa bytes) 2024-09-18T00:58:01.225 INFO:teuthology.orchestra.run.smithi042.stdout:980 punch from 0x656b6b1 to 0x6575ec4, (0xa813 bytes) 2024-09-18T00:58:01.225 INFO:teuthology.orchestra.run.smithi042.stdout:981 read 0x8802fd1 thru 0x880f52c (0xc55c bytes) 2024-09-18T00:58:01.225 INFO:teuthology.orchestra.run.smithi042.stdout:982 writesame 0x5b7aaa3 thru 0x5b8046b data_size 0x1(0x59c9 bytes) 2024-09-18T00:58:01.226 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:01.240 INFO:teuthology.orchestra.run.smithi042.stdout:983 writesame 0xca29a8e thru 0xca3712e data_size 0x87(0xd6a1 bytes) 2024-09-18T00:58:01.244 INFO:teuthology.orchestra.run.smithi042.stdout:984 read 0x7b702ed thru 0x7b72d65 (0x2a79 bytes) 2024-09-18T00:58:01.244 INFO:teuthology.orchestra.run.smithi042.stdout:986 write 0xd5fef4 thru 0xd69704 (0x9811 bytes) 2024-09-18T00:58:01.262 INFO:teuthology.orchestra.run.smithi042.stdout:988 write 0xacaabf4 thru 0xacb2e3f (0x824c bytes) 2024-09-18T00:58:01.270 INFO:teuthology.orchestra.run.smithi042.stdout:990 write 0xb7f15c1 thru 0xb7f7b7c (0x65bc bytes) 2024-09-18T00:58:01.281 INFO:teuthology.orchestra.run.smithi042.stdout:991 write 0x9b84bc4 thru 0x9b8fab1 (0xaeee bytes) 2024-09-18T00:58:01.289 INFO:teuthology.orchestra.run.smithi042.stdout:993 write 0x2f74ba2 thru 0x2f7a5f3 (0x5a52 bytes) 2024-09-18T00:58:01.297 INFO:teuthology.orchestra.run.smithi042.stdout:994 read 0x6dc60be thru 0x6dcb3bc (0x52ff bytes) 2024-09-18T00:58:01.297 INFO:teuthology.orchestra.run.smithi042.stdout:995 writesame 0x67781ec thru 0x677a913 data_size 0x1c(0x2728 bytes) 2024-09-18T00:58:01.304 INFO:teuthology.orchestra.run.smithi042.stdout:997 write 0x1320f5e thru 0x1327c59 (0x6cfc bytes) 2024-09-18T00:58:01.317 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:01.314+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:01.322 INFO:teuthology.orchestra.run.smithi042.stdout:998 clone 12 order 18 su 16384 sc 4 2024-09-18T00:58:01.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:01.394 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:01.395 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:01.408 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:01.416 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:01.410+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:01.491 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:01.507 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:01.507 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:01.517 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:01.510+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:01.524 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:01.524 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:01.540 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:01.540 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:01.561 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:01.561 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:01.571 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:01.571 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:01.581 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:01.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:01.586 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:01.602 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:01.602 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:01.617 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:01.614+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:01.621 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:01.621 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:01.718 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:01.714+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:01.724 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:01.724 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:01.724 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:01.724 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:01.738 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:01.819 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:01.814+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:01.855 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:01.855 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:01.856 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:01.856 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:01.858 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:58:01.858 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T00:58:01.870 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:01.962 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:01.958+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:02.021 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:02.018+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:02.045 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:02.045 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:02.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:02.046 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:02.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.046 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3710 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.046 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:02.046 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.175207872000001, 2024-09-18T00:58:02.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024498702000000001, 2024-09-18T00:58:02.046 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3710 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:02.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:02.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:58:02.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867971+0000", 2024-09-18T00:58:02.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.671e-06 2024-09-18T00:58:02.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867976+0000", 2024-09-18T00:58:02.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7030000000000003e-06 2024-09-18T00:58:02.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867981+0000", 2024-09-18T00:58:02.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8520000000000003e-06 2024-09-18T00:58:02.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892175+0000", 2024-09-18T00:58:02.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024193538000000001 2024-09-18T00:58:02.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892279+0000", 2024-09-18T00:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104017 2024-09-18T00:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892467+0000", 2024-09-18T00:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018858699999999999 2024-09-18T00:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3711 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.175189551999999, 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024701471999999999, 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3711 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6199999999999999e-07 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867989+0000", 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867995+0000", 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.127e-06 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868000+0000", 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892500+0000", 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024499725999999999 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892587+0000", 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7232999999999996e-05 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892688+0000", 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010115000000000001 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3712 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.175170594000001, 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024914385000000001, 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3712 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.057 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:02.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868008+0000", 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868014+0000", 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868019+0000", 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892727+0000", 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024707638000000001 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892817+0000", 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9569999999999998e-05 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892920+0000", 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103744 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3713 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.175151063000001, 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025115858000000001, 2024-09-18T00:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3713 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868026+0000", 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.58e-07 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868032+0000", 2024-09-18T00:58:02.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3790000000000003e-06 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868039+0000", 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6969999999999996e-06 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868044+0000", 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7060000000000003e-06 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892954+0000", 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024910704999999998 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893041+0000", 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6747000000000001e-05 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893141+0000", 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010026600000000001 2024-09-18T00:58:02.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3714 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.175126255999999, 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025291557999999999, 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3714 2024-09-18T00:58:02.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.068 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:58:02.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868051+0000", 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868052+0000", 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.863e-06 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868059+0000", 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2720000000000003e-06 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868064+0000", 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8860000000000003e-06 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893175+0000", 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025111114 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893262+0000", 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7476000000000001e-05 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893342+0000", 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9577000000000001e-05 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3715 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.175106751000001, 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025471191000000001, 2024-09-18T00:58:02.072 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3715 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2799999999999996e-07 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868072+0000", 2024-09-18T00:58:02.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9269999999999999e-06 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868078+0000", 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9830000000000001e-06 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868083+0000", 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.972e-06 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893373+0000", 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025290316 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893463+0000", 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9078000000000003e-05 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893541+0000", 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8386999999999993e-05 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.076 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3716 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.175087692999998, 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025664781000000001, 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3716 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2099999999999998e-07 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868091+0000", 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8950000000000001e-06 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868097+0000", 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1929999999999998e-06 2024-09-18T00:58:02.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868102+0000", 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6789999999999996e-06 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893570+0000", 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025468227999999999 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893657+0000", 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7176999999999996e-05 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893754+0000", 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6287999999999997e-05 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3717 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.081 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.175068586999998, 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025815932999999999, 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3717 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868111+0000", 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6860000000000002e-06 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868117+0000", 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868122+0000", 2024-09-18T00:58:02.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.69e-06 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893802+0000", 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025680397000000001 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893869+0000", 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6597e-05 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893924+0000", 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5031999999999998e-05 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3718 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.175047565, 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026018052, 2024-09-18T00:58:02.086 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3718 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:02.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-18T00:58:02.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868131+0000", 2024-09-18T00:58:02.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9369999999999998e-06 2024-09-18T00:58:02.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868138+0000", 2024-09-18T00:58:02.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4389999999999997e-06 2024-09-18T00:58:02.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868143+0000", 2024-09-18T00:58:02.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1440000000000002e-06 2024-09-18T00:58:02.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893973+0000", 2024-09-18T00:58:02.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025830490000000001 2024-09-18T00:58:02.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894063+0000", 2024-09-18T00:58:02.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9313000000000004e-05 2024-09-18T00:58:02.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894147+0000", 2024-09-18T00:58:02.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4342e-05 2024-09-18T00:58:02.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.091 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.092 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3719 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.092 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:02.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.175027609000001, 2024-09-18T00:58:02.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026147934000000001, 2024-09-18T00:58:02.092 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3719 2024-09-18T00:58:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-18T00:58:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868151+0000", 2024-09-18T00:58:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:58:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868157+0000", 2024-09-18T00:58:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868162+0000", 2024-09-18T00:58:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5619999999999997e-06 2024-09-18T00:58:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894183+0000", 2024-09-18T00:58:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026021083 2024-09-18T00:58:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894241+0000", 2024-09-18T00:58:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8505999999999999e-05 2024-09-18T00:58:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894297+0000", 2024-09-18T00:58:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5396000000000003e-05 2024-09-18T00:58:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3720 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:58:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.175009092, 2024-09-18T00:58:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026376077000000001, 2024-09-18T00:58:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3720 2024-09-18T00:58:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:58:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:58:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868168+0000", 2024-09-18T00:58:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:58:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868170+0000", 2024-09-18T00:58:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0889999999999998e-06 2024-09-18T00:58:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868176+0000", 2024-09-18T00:58:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3090000000000001e-06 2024-09-18T00:58:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868181+0000", 2024-09-18T00:58:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8810000000000004e-06 2024-09-18T00:58:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894341+0000", 2024-09-18T00:58:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026159636 2024-09-18T00:58:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894430+0000", 2024-09-18T00:58:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8986000000000005e-05 2024-09-18T00:58:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894543+0000", 2024-09-18T00:58:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011382700000000001 2024-09-18T00:58:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3721 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.174989574000001, 2024-09-18T00:58:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569722, 2024-09-18T00:58:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3721 2024-09-18T00:58:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:58:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868189+0000", 2024-09-18T00:58:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7460000000000001e-06 2024-09-18T00:58:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868195+0000", 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8490000000000002e-06 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868200+0000", 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1669999999999998e-06 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894623+0000", 2024-09-18T00:58:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02642338 2024-09-18T00:58:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894688+0000", 2024-09-18T00:58:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4183000000000005e-05 2024-09-18T00:58:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894757+0000", 2024-09-18T00:58:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9062999999999999e-05 2024-09-18T00:58:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3722 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.174970889000001, 2024-09-18T00:58:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026735041000000001, 2024-09-18T00:58:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3722 2024-09-18T00:58:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-18T00:58:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868208+0000", 2024-09-18T00:58:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8470000000000001e-06 2024-09-18T00:58:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868214+0000", 2024-09-18T00:58:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1090000000000003e-06 2024-09-18T00:58:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868218+0000", 2024-09-18T00:58:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.476e-06 2024-09-18T00:58:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894788+0000", 2024-09-18T00:58:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569274 2024-09-18T00:58:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894868+0000", 2024-09-18T00:58:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0711999999999996e-05 2024-09-18T00:58:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894941+0000", 2024-09-18T00:58:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:58:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3723 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.174952473000001, 2024-09-18T00:58:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026971922999999998, 2024-09-18T00:58:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3723 2024-09-18T00:58:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-07 2024-09-18T00:58:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868226+0000", 2024-09-18T00:58:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.812e-06 2024-09-18T00:58:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868232+0000", 2024-09-18T00:58:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7910000000000001e-06 2024-09-18T00:58:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868237+0000", 2024-09-18T00:58:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7339999999999999e-06 2024-09-18T00:58:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895056+0000", 2024-09-18T00:58:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026819195000000001 2024-09-18T00:58:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895103+0000", 2024-09-18T00:58:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7264e-05 2024-09-18T00:58:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895196+0000", 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2802999999999999e-05 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3724 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.174934261000001, 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027181833999999998, 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3724 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868243+0000", 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868245+0000", 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1160000000000002e-06 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868251+0000", 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1410000000000002e-06 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868256+0000", 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0130000000000001e-06 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895249+0000", 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02699352 2024-09-18T00:58:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895338+0000", 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8331000000000005e-05 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895424+0000", 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6348999999999996e-05 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3725 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.174915126999998, 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027296051000000002, 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3725 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868262+0000", 2024-09-18T00:58:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868267+0000", 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868274+0000", 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2360000000000001e-06 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868278+0000", 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7620000000000003e-06 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895458+0000", 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027179608000000001 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895511+0000", 2024-09-18T00:58:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2738000000000002e-05 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895557+0000", 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6631999999999997e-05 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3727 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:58:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.171604576, 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024398075000000002, 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3727 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:58:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871573+0000", 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.367e-06 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871580+0000", 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8029999999999999e-06 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871600+0000", 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0213000000000001e-05 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871615+0000", 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.471e-05 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895805+0000", 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024189677 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895923+0000", 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118496 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895970+0000", 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6808999999999999e-05 2024-09-18T00:58:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3728 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.171543317000001, 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024637071999999999, 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3728 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871634+0000", 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.074e-06 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871642+0000", 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8830000000000006e-06 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871660+0000", 2024-09-18T00:58:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8283000000000001e-05 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871692+0000", 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1714000000000002e-05 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896004+0000", 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024312099 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896099+0000", 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5036000000000003e-05 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896270+0000", 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017098300000000001 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.979211458, 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:58:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:58:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15723 3.f 3:f7b50ba2:::rbd_data.117c53e529c1.0000000000000058:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:58:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.8209425320000001, 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026056645999999999, 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15723 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:58:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222235+0000", 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222242+0000", 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4710000000000004e-06 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222255+0000", 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2877999999999999e-05 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222261+0000", 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.489e-06 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222328+0000", 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7280999999999995e-05 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222502+0000", 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000173609 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.227120+0000", 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046180129999999998 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248104+0000", 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020983979 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248166+0000", 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2340000000000003e-05 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248236+0000", 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9227000000000007e-05 2024-09-18T00:58:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248291+0000", 2024-09-18T00:58:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5019000000000003e-05 2024-09-18T00:58:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.151 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:02.151 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:58:02.153 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:02.154 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:02.146+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:02.191 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone11 from 0xca3928a (overlap 0x1a3773a) to 0x7dd6f7 2024-09-18T00:58:02.279 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:02.274+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:02.355 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:02.350+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:02.389 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:02.389 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:02.389 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:02.389 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:02.406 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:02.413 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-18T00:58:02.427 INFO:teuthology.orchestra.run.smithi042.stdout:999 write 0xb4bce9e thru 0xb4c60f8 (0x925b bytes) 2024-09-18T00:58:02.440 INFO:teuthology.orchestra.run.smithi042.stdout:1001 punch from 0xb048be8 to 0xb0512ff, (0x8717 bytes) 2024-09-18T00:58:02.441 INFO:teuthology.orchestra.run.smithi042.stdout:1003 read 0x90c810 thru 0x91b156 (0xe947 bytes) 2024-09-18T00:58:02.443 INFO:teuthology.orchestra.run.smithi042.stdout:1004 write 0x8ed01ea thru 0x8ed0369 (0x180 bytes) 2024-09-18T00:58:02.450 INFO:teuthology.orchestra.run.smithi042.stdout:1005 read 0x5ab64c9 thru 0x5ab945f (0x2f97 bytes) 2024-09-18T00:58:02.450 INFO:teuthology.orchestra.run.smithi042.stdout:1006 write 0xc125738 thru 0xc134f99 (0xf862 bytes) 2024-09-18T00:58:02.456 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:02.450+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:02.466 INFO:teuthology.orchestra.run.smithi042.stdout:1007 trunc from 0xca3928a to 0xa85d458 2024-09-18T00:58:02.491 INFO:teuthology.orchestra.run.smithi042.stdout:1008 read 0x20c734e thru 0x20ce953 (0x7606 bytes) 2024-09-18T00:58:02.491 INFO:teuthology.orchestra.run.smithi042.stdout:1009 writesame 0x61c6980 thru 0x61d199a data_size 0x1(0xb01b bytes) 2024-09-18T00:58:02.528 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:02.528 INFO:teuthology.orchestra.run.smithi042.stdout:1011 read 0x6cfbc9d thru 0x6d0a752 (0xeab6 bytes) 2024-09-18T00:58:02.529 INFO:teuthology.orchestra.run.smithi042.stdout:1012 read 0x39f98cc thru 0x39fc6af (0x2de4 bytes) 2024-09-18T00:58:02.529 INFO:teuthology.orchestra.run.smithi042.stdout:1013 read 0x8b784a7 thru 0x8b82542 (0xa09c bytes) 2024-09-18T00:58:02.529 INFO:teuthology.orchestra.run.smithi042.stdout:1015 read 0x79b2c88 thru 0x79c10e9 (0xe462 bytes) 2024-09-18T00:58:02.529 INFO:teuthology.orchestra.run.smithi042.stdout:1016 flatten 2024-09-18T00:58:02.557 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:02.550+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:02.568 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:02.568 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:02.568 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:02.568 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:02.568 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:02.586 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:02.658 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:02.654+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:02.708 INFO:teuthology.orchestra.run.smithi042.stdout:1017 write 0x6e2fc46 thru 0x6e3760e (0x79c9 bytes) 2024-09-18T00:58:02.714 INFO:teuthology.orchestra.run.smithi042.stdout:1018 punch from 0xa25fbdb to 0xa2681d9, (0x85fe bytes) 2024-09-18T00:58:02.714 INFO:teuthology.orchestra.run.smithi042.stdout:1019 write 0x1878ee4 thru 0x1881f9f (0x90bc bytes) 2024-09-18T00:58:02.719 INFO:teuthology.orchestra.run.smithi042.stdout:1022 read 0x273839f thru 0x273c381 (0x3fe3 bytes) 2024-09-18T00:58:02.729 INFO:teuthology.orchestra.run.smithi042.stdout:1023 write 0xba6669b thru 0xba73ce7 (0xd64d bytes) 2024-09-18T00:58:02.738 INFO:teuthology.orchestra.run.smithi042.stdout:1026 trunc from 0xba73ce8 to 0x943b33b 2024-09-18T00:58:02.758 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:02.754+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:02.759 INFO:teuthology.orchestra.run.smithi042.stdout:1027 trunc from 0x943b33b to 0x45b27a5 2024-09-18T00:58:02.778 INFO:teuthology.orchestra.run.smithi042.stdout:1028 write 0x11aa760 thru 0x11b38c7 (0x9168 bytes) 2024-09-18T00:58:02.791 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:58:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.500526120000004, 2024-09-18T00:58:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:58:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:58:02.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:58:02.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:58:02.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7423 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:02.799 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.30422757, 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045296661000000002, 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7423 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-18T00:58:02.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485621+0000", 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485627+0000", 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.156e-06 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485633+0000", 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789999999999999e-06 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530788+0000", 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045155001 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530853+0000", 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5634000000000002e-05 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530915+0000", 2024-09-18T00:58:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2022999999999999e-05 2024-09-18T00:58:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.804 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7424 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1146880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.304206580000001, 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045419355000000002, 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7424 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:02.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485642+0000", 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0640000000000001e-06 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485649+0000", 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485654+0000", 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2329999999999997e-06 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530941+0000", 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045286602000000002 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531004+0000", 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3455999999999997e-05 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531059+0000", 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4771000000000001e-05 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.304185067000001, 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:58:02.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:58:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.30416363, 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:58:02.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.304068302999999, 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:58:02.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:58:02.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:02.823 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.304042863999999, 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:58:02.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:58:02.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:58:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.304020340999999, 2024-09-18T00:58:02.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:02.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:58:02.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.303996901, 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:58:02.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.303973091, 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:58:02.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:02.842 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.303950726, 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:58:02.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.303924109, 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:58:02.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:58:02.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.299772338, 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:58:02.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.299751595, 2024-09-18T00:58:02.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:58:02.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:58:02.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.299706029999999, 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:58:02.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:58:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.299679689, 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:02.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.869 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.299657019, 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:58:02.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.874 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.299638646, 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:58:02.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:58:02.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.879 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.299618858000001, 2024-09-18T00:58:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:58:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:58:02.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:58:02.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:58:02.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:58:02.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.299591988, 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.888 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:02.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:02.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:02.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:58:02.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:58:02.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.892 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:02.892 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:02.892 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:02.893 INFO:teuthology.orchestra.run.smithi042.stdout:1029 write 0xed31bee thru 0xed3b1a5 (0x95b8 bytes) 2024-09-18T00:58:02.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:02.895 INFO:teuthology.orchestra.run.smithi042.stdout:1030 write 0x7e6d9dd thru 0x7e77b88 (0xa1ac bytes) 2024-09-18T00:58:02.895 INFO:teuthology.orchestra.run.smithi042.stdout:1031 punch from 0xb1880c2 to 0xb196698, (0xe5d6 bytes) 2024-09-18T00:58:02.895 INFO:teuthology.orchestra.run.smithi042.stdout:1033 trunc from 0xed3b1a6 to 0x353b5d4 2024-09-18T00:58:02.895 INFO:teuthology.orchestra.run.smithi042.stdout:1034 write 0xa223403 thru 0xa22e675 (0xb273 bytes) 2024-09-18T00:58:02.895 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:02.895 INFO:teuthology.orchestra.run.smithi042.stdout:1036 write 0x29b385c thru 0x29b6ae1 (0x3286 bytes) 2024-09-18T00:58:02.895 INFO:teuthology.orchestra.run.smithi042.stdout:1038 write 0x5f530fa thru 0x5f5edbd (0xbcc4 bytes) 2024-09-18T00:58:02.896 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:02.886+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:02.896 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xed58960 2024-09-18T00:58:02.901 INFO:teuthology.orchestra.run.smithi042.stdout:1039 trunc from 0xa22e676 to 0xed58960 2024-09-18T00:58:02.915 INFO:teuthology.orchestra.run.smithi042.stdout:1040 writesame 0x881aa4e thru 0x8822e34 data_size 0x1(0x83e7 bytes) 2024-09-18T00:58:02.930 INFO:teuthology.orchestra.run.smithi042.stdout:1041 punch from 0x148a6b2 to 0x1499745, (0xf093 bytes) 2024-09-18T00:58:02.930 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:02.930 INFO:teuthology.orchestra.run.smithi042.stdout:1043 read 0x8aa501a thru 0x8aaf359 (0xa340 bytes) 2024-09-18T00:58:02.930 INFO:teuthology.orchestra.run.smithi042.stdout:1044 read 0x2f2dc49 thru 0x2f39b5a (0xbf12 bytes) 2024-09-18T00:58:02.931 INFO:teuthology.orchestra.run.smithi042.stdout:1046 write 0x4568ed2 thru 0x456b536 (0x2665 bytes) 2024-09-18T00:58:02.936 INFO:teuthology.orchestra.run.smithi042.stdout:1048 punch from 0xc3ecc34 to 0xc3f11f7, (0x45c3 bytes) 2024-09-18T00:58:02.937 INFO:teuthology.orchestra.run.smithi042.stdout:1049 write 0x3fb724c thru 0x3fbee72 (0x7c27 bytes) 2024-09-18T00:58:02.960 INFO:teuthology.orchestra.run.smithi042.stdout:1050 trunc from 0xed58960 to 0x52ef9ff 2024-09-18T00:58:02.980 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:02.981 INFO:teuthology.orchestra.run.smithi042.stdout:1052 punch from 0x4e919bb to 0x4e952ac, (0x38f1 bytes) 2024-09-18T00:58:02.981 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:02.994 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:02.990+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:03.007 INFO:teuthology.orchestra.run.smithi042.stdout:1054 write 0xd12a5f2 thru 0xd130731 (0x6140 bytes) 2024-09-18T00:58:03.016 INFO:teuthology.orchestra.run.smithi042.stdout:1055 read 0xb6f2524 thru 0xb6ff4b6 (0xcf93 bytes) 2024-09-18T00:58:03.016 INFO:teuthology.orchestra.run.smithi042.stdout:1056 write 0xc636dce thru 0xc642edd (0xc110 bytes) 2024-09-18T00:58:03.024 INFO:teuthology.orchestra.run.smithi042.stdout:1057 write 0x3e2c0ba thru 0x3e3034c (0x4293 bytes) 2024-09-18T00:58:03.029 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:03.033 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T00:58:03.036 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:58:03.050 INFO:teuthology.orchestra.run.smithi042.stdout:1059 writesame 0xe705ef2 thru 0xe707026 data_size 0x1(0x1135 bytes) 2024-09-18T00:58:03.050 INFO:teuthology.orchestra.run.smithi042.stdout:1060 write 0xbfef86b thru 0xbfff25b (0xf9f1 bytes) 2024-09-18T00:58:03.061 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T00:58:03.069 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:03.069 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:03.069 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:03.069 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:03.070 INFO:teuthology.orchestra.run.smithi042.stdout:1061 trunc from 0xe707027 to 0x9a767f3 2024-09-18T00:58:03.089 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:03.107 INFO:teuthology.orchestra.run.smithi042.stdout:1062 write 0xbf8fb9f thru 0xbf9f2a0 (0xf702 bytes) 2024-09-18T00:58:03.107 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:03.102+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:03.115 INFO:teuthology.orchestra.run.smithi042.stdout:1063 write 0x256f45a thru 0x257d255 (0xddfc bytes) 2024-09-18T00:58:03.123 INFO:teuthology.orchestra.run.smithi042.stdout:1065 write 0xb79512e thru 0xb79e6fd (0x95d0 bytes) 2024-09-18T00:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout:1066 punch from 0x36c4db7 to 0x36c6429, (0x1672 bytes) 2024-09-18T00:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout:1067 write 0x9f2a327 thru 0x9f382ac (0xdf86 bytes) 2024-09-18T00:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout:1069 punch from 0x61e00bc to 0x61e875e, (0x86a2 bytes) 2024-09-18T00:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout:1072 writesame 0x6ea3ab0 thru 0x6ea5a27 data_size 0x35(0x1f78 bytes) 2024-09-18T00:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout:1073 trunc from 0xbf9f2a1 to 0x58fe6af 2024-09-18T00:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout:1074 read 0x1eda34a thru 0x1ede598 (0x424f bytes) 2024-09-18T00:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout:1075 read 0x12b10f0 thru 0x12bfbb8 (0xeac9 bytes) 2024-09-18T00:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout:1077 write 0x8f72bd5 thru 0x8f7cdc0 (0xa1ec bytes) 2024-09-18T00:58:03.195 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:03.190+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout:1079 writesame 0x545ae thru 0x5ae83 data_size 0xe(0x68d6 bytes) 2024-09-18T00:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout:1080 writesame 0x33043cd thru 0x3312875 data_size 0x1(0xe4a9 bytes) 2024-09-18T00:58:03.240 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:03.240 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:03.240 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:03.240 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:03.240 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:03.259 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:03.261 INFO:teuthology.orchestra.run.smithi042.stdout:1081 writesame 0xe8863b0 thru 0xe88fce7 data_size 0x4(0x9938 bytes) 2024-09-18T00:58:03.261 INFO:teuthology.orchestra.run.smithi042.stdout:1082 write 0xdde3c6b thru 0xddeee77 (0xb20d bytes) 2024-09-18T00:58:03.267 INFO:teuthology.orchestra.run.smithi042.stdout:1084 read 0x1680c09 thru 0x1689176 (0x856e bytes) 2024-09-18T00:58:03.268 INFO:teuthology.orchestra.run.smithi042.stdout:1085 writesame 0xb2812a6 thru 0xb2820a0 data_size 0x1(0xdfb bytes) 2024-09-18T00:58:03.272 INFO:teuthology.orchestra.run.smithi042.stdout:1086 read 0x3e19258 thru 0x3e26491 (0xd23a bytes) 2024-09-18T00:58:03.293 INFO:teuthology.orchestra.run.smithi042.stdout:1088 trunc from 0xe88fce8 to 0x5f4f41 2024-09-18T00:58:03.297 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:03.290+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:03.328 INFO:teuthology.orchestra.run.smithi042.stdout:1090 read 0x55f46f thru 0x55fda5 (0x937 bytes) 2024-09-18T00:58:03.341 INFO:teuthology.orchestra.run.smithi042.stdout:1091 write 0x348972c thru 0x348e26d (0x4b42 bytes) 2024-09-18T00:58:03.352 INFO:teuthology.orchestra.run.smithi042.stdout:1092 trunc from 0x348e26e to 0x837ee93 2024-09-18T00:58:03.363 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:03.363 INFO:teuthology.orchestra.run.smithi042.stdout:1098 punch from 0x4f3d6a6 to 0x4f45bdd, (0x8537 bytes) 2024-09-18T00:58:03.363 INFO:teuthology.orchestra.run.smithi042.stdout:1099 read 0x8134944 thru 0x8144160 (0xf81d bytes) 2024-09-18T00:58:03.379 INFO:teuthology.orchestra.run.smithi042.stdout:1101 write 0xd129079 thru 0xd12e589 (0x5511 bytes) 2024-09-18T00:58:03.386 INFO:teuthology.orchestra.run.smithi042.stdout:1102 punch from 0xb9fe08 to 0xba82e5, (0x84dd bytes) 2024-09-18T00:58:03.386 INFO:teuthology.orchestra.run.smithi042.stdout:1103 write 0x22c1c82 thru 0x22ca838 (0x8bb7 bytes) 2024-09-18T00:58:03.393 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:03.393 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:03.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:03.393 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.314749167999999, 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:58:03.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.313819577, 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.398 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.402 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.313805516999999, 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:58:03.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.312514942, 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.407 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:58:03.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.312494470000001, 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.309700017000001, 2024-09-18T00:58:03.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:58:03.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:58:03.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:58:03.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:58:03.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:58:03.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:58:03.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:58:03.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.421 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.421 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.421 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:03.421 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.309684832999999, 2024-09-18T00:58:03.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:03.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:58:03.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.309661887000001, 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:58:03.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:58:03.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1096 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2088960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.305499548, 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058813730000000002, 2024-09-18T00:58:03.430 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1096 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086204+0000", 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.75e-07 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086206+0000", 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.677e-06 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086210+0000", 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5950000000000001e-06 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086214+0000", 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4979999999999998e-06 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.144798+0000", 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058584587 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.144893+0000", 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4413000000000006e-05 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145017+0000", 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000124485 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.305484247999999, 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.305463030999999, 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:58:03.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:58:03.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.305441647999999, 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:58:03.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:58:03.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.447 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.447 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.447 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:03.447 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.301541793999998, 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:58:03.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.451 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.301527592999999, 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:58:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.456 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.301509555999999, 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:58:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:58:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.301495490000001, 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:58:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:58:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.297470139000001, 2024-09-18T00:58:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:58:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.297456127, 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:58:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.297442162999999, 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:58:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.535584209, 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:03.483 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:1104 read 0x129ade8 thru 0x12a68b1 (0xbaca bytes) 2024-09-18T00:58:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:1105 punch from 0xcfd3a94 to 0xcfe196b, (0xded7 bytes) 2024-09-18T00:58:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:1107 read 0x7835f66 thru 0x783b74b (0x57e6 bytes) 2024-09-18T00:58:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:1108 read 0x345ebcf thru 0x346d288 (0xe6ba bytes) 2024-09-18T00:58:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:1109 read 0x4e40737 thru 0x4e5050d (0xfdd7 bytes) 2024-09-18T00:58:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:1110 clone 13 order 18 su 32768 sc 8 2024-09-18T00:58:03.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:03.486 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:03.478+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:03.585 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:03.578+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:03.641 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:03.656 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:03.656 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:03.656 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:03.656 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:03.670 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:03.686 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:03.682+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:03.779 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:03.786 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:03.782+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:03.795 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:03.811 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:03.848 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:03.850 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:03.868 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:03.868 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:03.887 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:03.882+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:03.889 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:03.910 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:03.911 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:03.974 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:03.974 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:03.987 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:03.988 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:03.982+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:04.045 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T00:58:04.067 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T00:58:04.114 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:04.110+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:04.125 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:04.125 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:04.141 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:04.189 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:04.182+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:04.259 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:04.259 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:04.273 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:04.321 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:04.318+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:04.350 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-18T00:58:04.390 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:04.386+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:04.407 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:04.408 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:04.428 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:04.447 INFO:teuthology.orchestra.run.smithi042.stdout:1111 read 0x7d18ae thru 0x7da9ec (0x913f bytes) 2024-09-18T00:58:04.452 INFO:teuthology.orchestra.run.smithi042.stdout:1113 punch from 0xab627c9 to 0xab6cb9d, (0xa3d4 bytes) 2024-09-18T00:58:04.452 INFO:teuthology.orchestra.run.smithi042.stdout:1114 punch from 0x8602809 to 0x860ca3e, (0xa235 bytes) 2024-09-18T00:58:04.452 INFO:teuthology.orchestra.run.smithi042.stdout:1116 writesame 0xc8a633 thru 0xc96784 data_size 0x46(0xc152 bytes) 2024-09-18T00:58:04.458 INFO:teuthology.orchestra.run.smithi042.stdout:1117 read 0xafd4e9f thru 0xafe4c9f (0xfe01 bytes) 2024-09-18T00:58:04.458 INFO:teuthology.orchestra.run.smithi042.stdout:1118 read 0xaa8494 thru 0xaabd99 (0x3906 bytes) 2024-09-18T00:58:04.458 INFO:teuthology.orchestra.run.smithi042.stdout:1119 write 0x3707991 thru 0x370aa82 (0x30f2 bytes) 2024-09-18T00:58:04.465 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:04.465 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:04.465 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:04.465 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:04.465 INFO:teuthology.orchestra.run.smithi042.stdout:1126 write 0x5dba18b thru 0x5dbd251 (0x30c7 bytes) 2024-09-18T00:58:04.468 INFO:teuthology.orchestra.run.smithi042.stdout:1127 read 0x3978f61 thru 0x3985472 (0xc512 bytes) 2024-09-18T00:58:04.469 INFO:teuthology.orchestra.run.smithi042.stdout:1128 punch from 0x27277c to 0x277784, (0x5008 bytes) 2024-09-18T00:58:04.469 INFO:teuthology.orchestra.run.smithi042.stdout:1129 read 0xb539b3e thru 0xb53a4c3 (0x986 bytes) 2024-09-18T00:58:04.469 INFO:teuthology.orchestra.run.smithi042.stdout:1130 read 0x8f9f2fc thru 0x8fa506c (0x5d71 bytes) 2024-09-18T00:58:04.469 INFO:teuthology.orchestra.run.smithi042.stdout:1131 read 0x3e63822 thru 0x3e70c50 (0xd42f bytes) 2024-09-18T00:58:04.470 INFO:teuthology.orchestra.run.smithi042.stdout:1132 writesame 0x42ff339 thru 0x430c0e3 data_size 0x1(0xcdab bytes) 2024-09-18T00:58:04.507 INFO:teuthology.orchestra.run.smithi042.stdout:1133 trunc from 0xd12e58a to 0xd75fb7d 2024-09-18T00:58:04.517 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:04.514+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:04.530 INFO:teuthology.orchestra.run.smithi042.stdout:1134 trunc from 0xd75fb7d to 0x472e69c 2024-09-18T00:58:04.557 INFO:teuthology.orchestra.run.smithi042.stdout:1135 writesame 0x833c67 thru 0x83b5d4 data_size 0x42(0x796e bytes) 2024-09-18T00:58:04.577 INFO:teuthology.orchestra.run.smithi042.stdout:1137 write 0x6698180 thru 0x6699472 (0x12f3 bytes) 2024-09-18T00:58:04.582 INFO:teuthology.orchestra.run.smithi042.stdout:1139 punch from 0xafc9d5 to 0xafe9fe, (0x2029 bytes) 2024-09-18T00:58:04.582 INFO:teuthology.orchestra.run.smithi042.stdout:1140 punch from 0x63ec05a to 0x63f687b, (0xa821 bytes) 2024-09-18T00:58:04.591 INFO:teuthology.orchestra.run.smithi042.stdout:1141 trunc from 0x6699473 to 0x1e0517f 2024-09-18T00:58:04.591 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:04.586+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:04.598 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:04.598 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:04.615 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:04.636 INFO:teuthology.orchestra.run.smithi042.stdout:1143 trunc from 0x1e0517f to 0xeafbb04 2024-09-18T00:58:04.649 INFO:teuthology.orchestra.run.smithi042.stdout:1145 punch from 0x467e79e to 0x4680064, (0x18c6 bytes) 2024-09-18T00:58:04.649 INFO:teuthology.orchestra.run.smithi042.stdout:1147 write 0x9036d67 thru 0x903eb47 (0x7de1 bytes) 2024-09-18T00:58:04.656 INFO:teuthology.orchestra.run.smithi042.stdout:1148 writesame 0x41979fa thru 0x41a265a data_size 0x1(0xac61 bytes) 2024-09-18T00:58:04.687 INFO:teuthology.orchestra.run.smithi042.stdout:1149 write 0xb6e7f81 thru 0xb6f26ec (0xa76c bytes) 2024-09-18T00:58:04.698 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:04.699 INFO:teuthology.orchestra.run.smithi042.stdout:1152 punch from 0x160ad4f to 0x160fdf9, (0x50aa bytes) 2024-09-18T00:58:04.699 INFO:teuthology.orchestra.run.smithi042.stdout:1155 read 0x98836c4 thru 0x988d425 (0x9d62 bytes) 2024-09-18T00:58:04.699 INFO:teuthology.orchestra.run.smithi042.stdout:1156 read 0xb343eae thru 0xb347021 (0x3174 bytes) 2024-09-18T00:58:04.699 INFO:teuthology.orchestra.run.smithi042.stdout:1158 read 0x20fc38c thru 0x20fe0b5 (0x1d2a bytes) 2024-09-18T00:58:04.699 INFO:teuthology.orchestra.run.smithi042.stdout:1159 read 0x5cbbbb0 thru 0x5cbd4ec (0x193d bytes) 2024-09-18T00:58:04.699 INFO:teuthology.orchestra.run.smithi042.stdout:1160 write 0xd9f207 thru 0xda27b6 (0x35b0 bytes) 2024-09-18T00:58:04.701 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:04.698+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:04.707 INFO:teuthology.orchestra.run.smithi042.stdout:1161 write 0x49311e8 thru 0x4933753 (0x256c bytes) 2024-09-18T00:58:04.714 INFO:teuthology.orchestra.run.smithi042.stdout:1162 punch from 0x3a196a to 0x3a7438, (0x5ace bytes) 2024-09-18T00:58:04.714 INFO:teuthology.orchestra.run.smithi042.stdout:1163 writesame 0xcd38e63 thru 0xcd3aac0 data_size 0x1(0x1c5e bytes) 2024-09-18T00:58:04.724 INFO:teuthology.orchestra.run.smithi042.stdout:1164 punch from 0x5894861 to 0x58a2773, (0xdf12 bytes) 2024-09-18T00:58:04.726 INFO:teuthology.orchestra.run.smithi042.stdout:1165 trunc from 0xeafbb04 to 0xe0a0f7a 2024-09-18T00:58:04.738 INFO:teuthology.orchestra.run.smithi042.stdout:1166 punch from 0x1fdd8fb to 0x1fe5cc1, (0x83c6 bytes) 2024-09-18T00:58:04.738 INFO:teuthology.orchestra.run.smithi042.stdout:1167 write 0x48c08e7 thru 0x48c81e8 (0x7902 bytes) 2024-09-18T00:58:04.748 INFO:teuthology.orchestra.run.smithi042.stdout:1168 punch from 0xa8bab25 to 0xa8bf2e0, (0x47bb bytes) 2024-09-18T00:58:04.748 INFO:teuthology.orchestra.run.smithi042.stdout:1169 read 0x6e68460 thru 0x6e68cb1 (0x852 bytes) 2024-09-18T00:58:04.748 INFO:teuthology.orchestra.run.smithi042.stdout:1170 write 0x3781016 thru 0x37908f0 (0xf8db bytes) 2024-09-18T00:58:04.785 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:04.785 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:04.790 INFO:teuthology.orchestra.run.smithi042.stdout:1171 trunc from 0xe0a0f7a to 0x358c06f 2024-09-18T00:58:04.793 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:04.786+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:04.799 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:04.859 INFO:teuthology.orchestra.run.smithi042.stdout:1173 write 0xec53517 thru 0xec53acd (0x5b7 bytes) 2024-09-18T00:58:04.866 INFO:teuthology.orchestra.run.smithi042.stdout:1174 write 0x13d0cff thru 0x13e0bcd (0xfecf bytes) 2024-09-18T00:58:04.877 INFO:teuthology.orchestra.run.smithi042.stdout:1175 write 0xe4cff94 thru 0xe4d0b3c (0xba9 bytes) 2024-09-18T00:58:04.886 INFO:teuthology.orchestra.run.smithi042.stdout:1176 read 0xa8647ee thru 0xa86e3e2 (0x9bf5 bytes) 2024-09-18T00:58:04.886 INFO:teuthology.orchestra.run.smithi042.stdout:1177 write 0x2e53f01 thru 0x2e60e2c (0xcf2c bytes) 2024-09-18T00:58:04.894 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:04.890+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:04.896 INFO:teuthology.orchestra.run.smithi042.stdout:1178 read 0x67070fd thru 0x6713907 (0xc80b bytes) 2024-09-18T00:58:04.932 INFO:teuthology.orchestra.run.smithi042.stdout:1179 trunc from 0xec53ace to 0xc40893 2024-09-18T00:58:04.969 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:04.969 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:04.983 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:04.994 INFO:teuthology.orchestra.run.smithi042.stdout:1180 write 0x672509f thru 0x672aa03 (0x5965 bytes) 2024-09-18T00:58:05.002 INFO:teuthology.orchestra.run.smithi042.stdout:1181 read 0x29edfb8 thru 0x29f6a1f (0x8a68 bytes) 2024-09-18T00:58:05.006 INFO:teuthology.orchestra.run.smithi042.stdout:1182 trunc from 0x672aa04 to 0x539d53c 2024-09-18T00:58:05.029 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:05.026+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:05.048 INFO:teuthology.orchestra.run.smithi042.stdout:1184 writesame 0x858aa1a thru 0x85950fb data_size 0x29(0xa6e2 bytes) 2024-09-18T00:58:05.048 INFO:teuthology.orchestra.run.smithi042.stdout:1185 writesame 0x4c91b1a thru 0x4c91d82 data_size 0x1(0x269 bytes) 2024-09-18T00:58:05.055 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T00:58:05.061 INFO:teuthology.orchestra.run.smithi042.stdout:1186 read 0x621b44f thru 0x621c77f (0x1331 bytes) 2024-09-18T00:58:05.065 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:58:05.077 INFO:teuthology.orchestra.run.smithi042.stdout:1188 write 0xa714adc thru 0xa7243bf (0xf8e4 bytes) 2024-09-18T00:58:05.085 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T00:58:05.085 INFO:teuthology.orchestra.run.smithi042.stdout:1189 read 0x8d7cb07 thru 0x8d8c83e (0xfd38 bytes) 2024-09-18T00:58:05.092 INFO:teuthology.orchestra.run.smithi042.stdout:1190 trunc from 0xa7243c0 to 0xe8fbe05 2024-09-18T00:58:05.101 INFO:teuthology.orchestra.run.smithi042.stdout:1191 read 0xd40bcb2 thru 0xd416257 (0xa5a6 bytes) 2024-09-18T00:58:05.101 INFO:teuthology.orchestra.run.smithi042.stdout:1192 write 0xdfd108e thru 0xdfd877b (0x76ee bytes) 2024-09-18T00:58:05.101 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:05.094+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:05.107 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:05.107 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:05.121 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:05.127 INFO:teuthology.orchestra.run.smithi042.stdout:1193 trunc from 0xe8fbe05 to 0x1f853f5 2024-09-18T00:58:05.158 INFO:teuthology.orchestra.run.smithi042.stdout:1195 trunc from 0x1f853f5 to 0xc3726df 2024-09-18T00:58:05.166 INFO:teuthology.orchestra.run.smithi042.stdout:1196 read 0x231ab08 thru 0x2325697 (0xab90 bytes) 2024-09-18T00:58:05.166 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.166 INFO:teuthology.orchestra.run.smithi042.stdout:1199 punch from 0x9b8dc2e to 0x9b946ed, (0x6abf bytes) 2024-09-18T00:58:05.167 INFO:teuthology.orchestra.run.smithi042.stdout:1200 read 0x8ec0fb1 thru 0x8ec1b20 (0xb70 bytes) 2024-09-18T00:58:05.167 INFO:teuthology.orchestra.run.smithi042.stdout:1201 write 0x6bb2d25 thru 0x6bbbadd (0x8db9 bytes) 2024-09-18T00:58:05.171 INFO:teuthology.orchestra.run.smithi042.stdout:1203 read 0x57248ea thru 0x573259d (0xdcb4 bytes) 2024-09-18T00:58:05.171 INFO:teuthology.orchestra.run.smithi042.stdout:1205 write 0x419538 thru 0x41f98a (0x6453 bytes) 2024-09-18T00:58:05.182 INFO:teuthology.orchestra.run.smithi042.stdout:1206 trunc from 0xc3726df to 0x7fdf526 2024-09-18T00:58:05.189 INFO:teuthology.orchestra.run.smithi042.stdout:1207 write 0x24debcb thru 0x24eca28 (0xde5e bytes) 2024-09-18T00:58:05.195 INFO:teuthology.orchestra.run.smithi042.stdout:1208 read 0x166e4db thru 0x1670cfe (0x2824 bytes) 2024-09-18T00:58:05.202 INFO:teuthology.orchestra.run.smithi042.stdout:1209 writesame 0x842ac71 thru 0x842b063 data_size 0x1(0x3f3 bytes) 2024-09-18T00:58:05.202 INFO:teuthology.orchestra.run.smithi042.stdout:1211 write 0x67754f4 thru 0x677d543 (0x8050 bytes) 2024-09-18T00:58:05.207 INFO:teuthology.orchestra.run.smithi042.stdout:1216 punch from 0x2ac4c05 to 0x2ad0834, (0xbc2f bytes) 2024-09-18T00:58:05.208 INFO:teuthology.orchestra.run.smithi042.stdout:1217 punch from 0x11e1339 to 0x11e24dc, (0x11a3 bytes) 2024-09-18T00:58:05.208 INFO:teuthology.orchestra.run.smithi042.stdout:1218 punch from 0x55f52c1 to 0x55fe1c6, (0x8f05 bytes) 2024-09-18T00:58:05.208 INFO:teuthology.orchestra.run.smithi042.stdout:1219 read 0x68e93f thru 0x6946f7 (0x5db9 bytes) 2024-09-18T00:58:05.208 INFO:teuthology.orchestra.run.smithi042.stdout:1221 read 0x22d96fb thru 0x22e6b6b (0xd471 bytes) 2024-09-18T00:58:05.208 INFO:teuthology.orchestra.run.smithi042.stdout:1222 write 0xf533ff thru 0xf5e3b5 (0xafb7 bytes) 2024-09-18T00:58:05.209 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:05.206+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:05.213 INFO:teuthology.orchestra.run.smithi042.stdout:1223 punch from 0x749f747 to 0x74a181f, (0x20d8 bytes) 2024-09-18T00:58:05.213 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.213 INFO:teuthology.orchestra.run.smithi042.stdout:1225 punch from 0x590be06 to 0x59194dc, (0xd6d6 bytes) 2024-09-18T00:58:05.213 INFO:teuthology.orchestra.run.smithi042.stdout:1226 write 0x37c28ee thru 0x37c676f (0x3e82 bytes) 2024-09-18T00:58:05.216 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.216 INFO:teuthology.orchestra.run.smithi042.stdout:1229 write 0x612fe7d thru 0x613e05e (0xe1e2 bytes) 2024-09-18T00:58:05.232 INFO:teuthology.orchestra.run.smithi042.stdout:1230 write 0x9ec799c thru 0x9ecd531 (0x5b96 bytes) 2024-09-18T00:58:05.241 INFO:teuthology.orchestra.run.smithi042.stdout:1231 writesame 0x68c8097 thru 0x68c91c3 data_size 0x1(0x112d bytes) 2024-09-18T00:58:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xede93f8 2024-09-18T00:58:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:1232 trunc from 0x9ecd532 to 0xede93f8 2024-09-18T00:58:05.275 INFO:teuthology.orchestra.run.smithi042.stdout:1233 trunc from 0xede93f8 to 0x383f730 2024-09-18T00:58:05.290 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:05.290 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:05.293 INFO:teuthology.orchestra.run.smithi042.stdout:1234 trunc from 0x383f730 to 0x1e914b6 2024-09-18T00:58:05.296 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:05.290+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:05.306 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:05.306 INFO:teuthology.orchestra.run.smithi042.stdout:1235 read 0x1298024 thru 0x12a0751 (0x872e bytes) 2024-09-18T00:58:05.307 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.307 INFO:teuthology.orchestra.run.smithi042.stdout:1237 trunc from 0x1e914b6 to 0x16beceb 2024-09-18T00:58:05.314 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.314 INFO:teuthology.orchestra.run.smithi042.stdout:1241 punch from 0x47b566 to 0x489fdb, (0xea75 bytes) 2024-09-18T00:58:05.322 INFO:teuthology.orchestra.run.smithi042.stdout:1243 trunc from 0x16beceb to 0x6d7f0ba 2024-09-18T00:58:05.328 INFO:teuthology.orchestra.run.smithi042.stdout:1244 write 0x1fae7e4 thru 0x1faf51d (0xd3a bytes) 2024-09-18T00:58:05.338 INFO:teuthology.orchestra.run.smithi042.stdout:1245 trunc from 0x6d7f0ba to 0xa404849 2024-09-18T00:58:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:1246 punch from 0x59f2b8d to 0x59fb632, (0x8aa5 bytes) 2024-09-18T00:58:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:1247 writesame 0x3e20f09 thru 0x3e21955 data_size 0x3(0xa4d bytes) 2024-09-18T00:58:05.349 INFO:teuthology.orchestra.run.smithi042.stdout:1249 read 0x555a451 thru 0x555eb6f (0x471f bytes) 2024-09-18T00:58:05.362 INFO:teuthology.orchestra.run.smithi042.stdout:1250 writesame 0xd2b07b2 thru 0xd2b1c31 data_size 0x8(0x1480 bytes) 2024-09-18T00:58:05.362 INFO:teuthology.orchestra.run.smithi042.stdout:1251 punch from 0x99b8060 to 0x99c5a3e, (0xd9de bytes) 2024-09-18T00:58:05.362 INFO:teuthology.orchestra.run.smithi042.stdout:1252 punch from 0x31cd96c to 0x31d8fa0, (0xb634 bytes) 2024-09-18T00:58:05.362 INFO:teuthology.orchestra.run.smithi042.stdout:1254 read 0xa14787 thru 0xa1dd68 (0x95e2 bytes) 2024-09-18T00:58:05.362 INFO:teuthology.orchestra.run.smithi042.stdout:1255 read 0x7bfbf2a thru 0x7c06e77 (0xaf4e bytes) 2024-09-18T00:58:05.362 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.365 INFO:teuthology.orchestra.run.smithi042.stdout:1258 trunc from 0xd2b1c32 to 0xb81d2f5 2024-09-18T00:58:05.376 INFO:teuthology.orchestra.run.smithi042.stdout:1259 write 0xb11d9c1 thru 0xb125b0b (0x814b bytes) 2024-09-18T00:58:05.384 INFO:teuthology.orchestra.run.smithi042.stdout:1260 write 0x78b43af thru 0x78c1d95 (0xd9e7 bytes) 2024-09-18T00:58:05.391 INFO:teuthology.orchestra.run.smithi042.stdout:1262 writesame 0x1830be1 thru 0x1832a7d data_size 0x1(0x1e9d bytes) 2024-09-18T00:58:05.397 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:05.390+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:05.413 INFO:teuthology.orchestra.run.smithi042.stdout:1263 write 0xd6b6628 thru 0xd6b7b57 (0x1530 bytes) 2024-09-18T00:58:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:1265 read 0x7e0a2d8 thru 0x7e14dba (0xaae3 bytes) 2024-09-18T00:58:05.418 INFO:teuthology.orchestra.run.smithi042.stdout:1266 writesame 0x8a01471 thru 0x8a0f310 data_size 0x89(0xdea0 bytes) 2024-09-18T00:58:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:1267 punch from 0x11f052e to 0x11f1266, (0xd38 bytes) 2024-09-18T00:58:05.423 INFO:teuthology.orchestra.run.smithi042.stdout:1268 write 0x604182a thru 0x604edce (0xd5a5 bytes) 2024-09-18T00:58:05.429 INFO:teuthology.orchestra.run.smithi042.stdout:1270 read 0xb09a812 thru 0xb0a19dd (0x71cc bytes) 2024-09-18T00:58:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:1271 punch from 0x17c54a4 to 0x17cfd08, (0xa864 bytes) 2024-09-18T00:58:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:1273 punch from 0x6dda4e3 to 0x6ddc560, (0x207d bytes) 2024-09-18T00:58:05.430 INFO:teuthology.orchestra.run.smithi042.stdout:1274 writesame 0xc0ef136 thru 0xc0fe702 data_size 0x19(0xf5cd bytes) 2024-09-18T00:58:05.447 INFO:teuthology.orchestra.run.smithi042.stdout:1275 trunc from 0xd6b7b58 to 0x7acadfe 2024-09-18T00:58:05.478 INFO:teuthology.orchestra.run.smithi042.stdout:1276 writesame 0xbb2e736 thru 0xbb3a200 data_size 0x1(0xbacb bytes) 2024-09-18T00:58:05.481 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:05.482 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:05.493 INFO:teuthology.orchestra.run.smithi042.stdout:1277 write 0xeb6329b thru 0xeb643eb (0x1151 bytes) 2024-09-18T00:58:05.497 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:05.497 INFO:teuthology.orchestra.run.smithi042.stdout:1279 write 0xbb431e3 thru 0xbb4af49 (0x7d67 bytes) 2024-09-18T00:58:05.498 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:05.494+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:05.502 INFO:teuthology.orchestra.run.smithi042.stdout:1280 punch from 0x4bb3d5d to 0x4bb4530, (0x7d3 bytes) 2024-09-18T00:58:05.502 INFO:teuthology.orchestra.run.smithi042.stdout:1282 punch from 0x49e69e1 to 0x49e8910, (0x1f2f bytes) 2024-09-18T00:58:05.508 INFO:teuthology.orchestra.run.smithi042.stdout:1283 trunc from 0xeb643ec to 0xb625368 2024-09-18T00:58:05.536 INFO:teuthology.orchestra.run.smithi042.stdout:1284 write 0xd71f644 thru 0xd72684b (0x7208 bytes) 2024-09-18T00:58:05.544 INFO:teuthology.orchestra.run.smithi042.stdout:1285 write 0x1197f27 thru 0x11a4951 (0xca2b bytes) 2024-09-18T00:58:05.550 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:1289 punch from 0xb5afe76 to 0xb5bc15c, (0xc2e6 bytes) 2024-09-18T00:58:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:1291 punch from 0x3ff5b5a to 0x4000dac, (0xb252 bytes) 2024-09-18T00:58:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.551 INFO:teuthology.orchestra.run.smithi042.stdout:1294 write 0x4b0011b thru 0x4b01863 (0x1749 bytes) 2024-09-18T00:58:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:1295 read 0xcda9fc3 thru 0xcdb5cc1 (0xbcff bytes) 2024-09-18T00:58:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:1296 read 0x2ed1f20 thru 0x2ed2e2f (0xf10 bytes) 2024-09-18T00:58:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:1297 read 0x639f287 thru 0x63a8d38 (0x9ab2 bytes) 2024-09-18T00:58:05.555 INFO:teuthology.orchestra.run.smithi042.stdout:1298 write 0xb93f255 thru 0xb94c4a3 (0xd24f bytes) 2024-09-18T00:58:05.562 INFO:teuthology.orchestra.run.smithi042.stdout:1299 read 0xb0c3561 thru 0xb0ca06b (0x6b0b bytes) 2024-09-18T00:58:05.573 INFO:teuthology.orchestra.run.smithi042.stdout:1300 write 0xd872e83 thru 0xd876bbc (0x3d3a bytes) 2024-09-18T00:58:05.577 INFO:teuthology.orchestra.run.smithi042.stdout:1301 write 0x86c1051 thru 0x86cec87 (0xdc37 bytes) 2024-09-18T00:58:05.584 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.584 INFO:teuthology.orchestra.run.smithi042.stdout:1304 writesame 0x803385b thru 0x803a803 data_size 0x1(0x6fa9 bytes) 2024-09-18T00:58:05.598 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:05.594+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:05.600 INFO:teuthology.orchestra.run.smithi042.stdout:1306 write 0x4cdd364 thru 0x4ce60d0 (0x8d6d bytes) 2024-09-18T00:58:05.633 INFO:teuthology.orchestra.run.smithi042.stdout:1307 trunc from 0xd876bbd to 0x1fae047 2024-09-18T00:58:05.676 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:05.677 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:05.686 INFO:teuthology.orchestra.run.smithi042.stdout:1309 write 0x6327f89 thru 0x632c888 (0x4900 bytes) 2024-09-18T00:58:05.691 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:05.699 INFO:teuthology.orchestra.run.smithi042.stdout:1310 trunc from 0x632c889 to 0x822628f 2024-09-18T00:58:05.709 INFO:teuthology.orchestra.run.smithi042.stdout:1311 punch from 0x4aad15 to 0x4b981b, (0xeb06 bytes) 2024-09-18T00:58:05.709 INFO:teuthology.orchestra.run.smithi042.stdout:1312 write 0x6f07e09 thru 0x6f09b7d (0x1d75 bytes) 2024-09-18T00:58:05.717 INFO:teuthology.orchestra.run.smithi042.stdout:1313 punch from 0x156cec to 0x1618a6, (0xabba bytes) 2024-09-18T00:58:05.722 INFO:teuthology.orchestra.run.smithi042.stdout:1314 trunc from 0x822628f to 0xa24a6e7 2024-09-18T00:58:05.733 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.733 INFO:teuthology.orchestra.run.smithi042.stdout:1316 read 0x684c11e thru 0x685713c (0xb01f bytes) 2024-09-18T00:58:05.733 INFO:teuthology.orchestra.run.smithi042.stdout:1317 read 0x3a22ebd thru 0x3a2abaf (0x7cf3 bytes) 2024-09-18T00:58:05.733 INFO:teuthology.orchestra.run.smithi042.stdout:1319 punch from 0x7d96223 to 0x7da43f7, (0xe1d4 bytes) 2024-09-18T00:58:05.733 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.733 INFO:teuthology.orchestra.run.smithi042.stdout:1321 write 0x21de58f thru 0x21e642c (0x7e9e bytes) 2024-09-18T00:58:05.734 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:05.730+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:05.753 INFO:teuthology.orchestra.run.smithi042.stdout:1322 trunc from 0xa24a6e7 to 0x4120557 2024-09-18T00:58:05.774 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:05.774 INFO:teuthology.orchestra.run.smithi042.stdout:1324 write 0x1daf9cb thru 0x1dbc09a (0xc6d0 bytes) 2024-09-18T00:58:05.789 INFO:teuthology.orchestra.run.smithi042.stdout:1325 trunc from 0x4120557 to 0x77d2dc4 2024-09-18T00:58:05.800 INFO:teuthology.orchestra.run.smithi042.stdout:1326 write 0x27848f0 thru 0x2793fed (0xf6fe bytes) 2024-09-18T00:58:05.801 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:05.794+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:05.829 INFO:teuthology.orchestra.run.smithi042.stdout:1327 writesame 0xa708a04 thru 0xa70a81f data_size 0x2f(0x1e1c bytes) 2024-09-18T00:58:05.829 INFO:teuthology.orchestra.run.smithi042.stdout:1328 read 0x767035a thru 0x7674a61 (0x4708 bytes) 2024-09-18T00:58:05.829 INFO:teuthology.orchestra.run.smithi042.stdout:1329 clone 14 order 21 su 1048576 sc 3 2024-09-18T00:58:05.855 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:05.855 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:05.869 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:05.917 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:05.914+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:05.928 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:06.002 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:05.998+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:06.039 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:06.039 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:06.054 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:06.059 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:58:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T00:58:06.098 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T00:58:06.103 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:06.098+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:06.106 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:06.106 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:06.126 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:06.146 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:06.146 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:06.162 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone13 from 0xa70a820 (overlap 0xc40893) to 0xb06e1d 2024-09-18T00:58:06.163 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:06.164 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:06.191 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:06.191 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:06.200 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:06.200 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:06.200 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:06.200 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:06.204 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:06.198+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:06.218 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:06.220 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:06.221 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:06.246 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:06.246 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:06.281 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:06.281 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:06.282 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-18T00:58:06.305 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:06.302+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:06.346 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:06.346 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:06.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:06.406 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:06.402+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:06.507 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:06.502+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:06.532 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:06.533 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:06.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:06.533 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:06.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.533 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3710 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.533 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:06.533 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661442982000001, 2024-09-18T00:58:06.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024498702000000001, 2024-09-18T00:58:06.533 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3710 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:06.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867971+0000", 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.671e-06 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867976+0000", 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7030000000000003e-06 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867981+0000", 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8520000000000003e-06 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892175+0000", 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024193538000000001 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892279+0000", 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104017 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892467+0000", 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018858699999999999 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.537 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3711 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661424662000002, 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024701471999999999, 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3711 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6199999999999999e-07 2024-09-18T00:58:06.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867989+0000", 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867995+0000", 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.127e-06 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868000+0000", 2024-09-18T00:58:06.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892500+0000", 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024499725999999999 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892587+0000", 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7232999999999996e-05 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892688+0000", 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010115000000000001 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3712 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661405704, 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024914385000000001, 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3712 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868008+0000", 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868014+0000", 2024-09-18T00:58:06.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868019+0000", 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892727+0000", 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024707638000000001 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892817+0000", 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9569999999999998e-05 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892920+0000", 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103744 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3713 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661386173, 2024-09-18T00:58:06.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025115858000000001, 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3713 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:58:06.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868026+0000", 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.58e-07 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868032+0000", 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3790000000000003e-06 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868039+0000", 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6969999999999996e-06 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868044+0000", 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7060000000000003e-06 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892954+0000", 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024910704999999998 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893041+0000", 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6747000000000001e-05 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893141+0000", 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010026600000000001 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3714 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661361366000001, 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025291557999999999, 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3714 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868051+0000", 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868052+0000", 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.863e-06 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868059+0000", 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2720000000000003e-06 2024-09-18T00:58:06.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868064+0000", 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8860000000000003e-06 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893175+0000", 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025111114 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893262+0000", 2024-09-18T00:58:06.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7476000000000001e-05 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893342+0000", 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9577000000000001e-05 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3715 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:06.555 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661341861, 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025471191000000001, 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3715 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:06.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2799999999999996e-07 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868072+0000", 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9269999999999999e-06 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868078+0000", 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9830000000000001e-06 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868083+0000", 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.972e-06 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893373+0000", 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025290316 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893463+0000", 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9078000000000003e-05 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893541+0000", 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8386999999999993e-05 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3716 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661322803000001, 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025664781000000001, 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3716 2024-09-18T00:58:06.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2099999999999998e-07 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868091+0000", 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8950000000000001e-06 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868097+0000", 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1929999999999998e-06 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868102+0000", 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6789999999999996e-06 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893570+0000", 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025468227999999999 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893657+0000", 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7176999999999996e-05 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893754+0000", 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6287999999999997e-05 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3717 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661303697000001, 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025815932999999999, 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3717 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868111+0000", 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6860000000000002e-06 2024-09-18T00:58:06.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868117+0000", 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868122+0000", 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.69e-06 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893802+0000", 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025680397000000001 2024-09-18T00:58:06.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893869+0000", 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6597e-05 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893924+0000", 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5031999999999998e-05 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3718 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661282674999999, 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026018052, 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3718 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.569 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:06.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868131+0000", 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9369999999999998e-06 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868138+0000", 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4389999999999997e-06 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868143+0000", 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1440000000000002e-06 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893973+0000", 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025830490000000001 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894063+0000", 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9313000000000004e-05 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894147+0000", 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4342e-05 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3719 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661262719, 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026147934000000001, 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3719 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868151+0000", 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868157+0000", 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868162+0000", 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5619999999999997e-06 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894183+0000", 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026021083 2024-09-18T00:58:06.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894241+0000", 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8505999999999999e-05 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894297+0000", 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5396000000000003e-05 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3720 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661244201999999, 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026376077000000001, 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3720 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868168+0000", 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:58:06.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868170+0000", 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0889999999999998e-06 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868176+0000", 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3090000000000001e-06 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868181+0000", 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8810000000000004e-06 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894341+0000", 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026159636 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894430+0000", 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8986000000000005e-05 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894543+0000", 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011382700000000001 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3721 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661224684, 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569722, 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3721 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:06.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868189+0000", 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7460000000000001e-06 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868195+0000", 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8490000000000002e-06 2024-09-18T00:58:06.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868200+0000", 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1669999999999998e-06 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894623+0000", 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02642338 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894688+0000", 2024-09-18T00:58:06.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4183000000000005e-05 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894757+0000", 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9062999999999999e-05 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3722 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661205999, 2024-09-18T00:58:06.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026735041000000001, 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3722 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868208+0000", 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8470000000000001e-06 2024-09-18T00:58:06.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868214+0000", 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1090000000000003e-06 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868218+0000", 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.476e-06 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894788+0000", 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569274 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894868+0000", 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0711999999999996e-05 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894941+0000", 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3723 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661187583, 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026971922999999998, 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3723 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.591 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-07 2024-09-18T00:58:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868226+0000", 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.812e-06 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868232+0000", 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7910000000000001e-06 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868237+0000", 2024-09-18T00:58:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7339999999999999e-06 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895056+0000", 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026819195000000001 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895103+0000", 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7264e-05 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895196+0000", 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2802999999999999e-05 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3724 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661169371, 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027181833999999998, 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3724 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868243+0000", 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868245+0000", 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1160000000000002e-06 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868251+0000", 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1410000000000002e-06 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868256+0000", 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0130000000000001e-06 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895249+0000", 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02699352 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895338+0000", 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8331000000000005e-05 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895424+0000", 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6348999999999996e-05 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3725 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:58:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.661150237000001, 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027296051000000002, 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3725 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868262+0000", 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868267+0000", 2024-09-18T00:58:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868274+0000", 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2360000000000001e-06 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868278+0000", 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7620000000000003e-06 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895458+0000", 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027179608000000001 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895511+0000", 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2738000000000002e-05 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895557+0000", 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6631999999999997e-05 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3727 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.657839685999999, 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024398075000000002, 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3727 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871573+0000", 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.367e-06 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871580+0000", 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8029999999999999e-06 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871600+0000", 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0213000000000001e-05 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871615+0000", 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.471e-05 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895805+0000", 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024189677 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895923+0000", 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118496 2024-09-18T00:58:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895970+0000", 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6808999999999999e-05 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3728 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.657778427, 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024637071999999999, 2024-09-18T00:58:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3728 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871634+0000", 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.074e-06 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871642+0000", 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8830000000000006e-06 2024-09-18T00:58:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871660+0000", 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8283000000000001e-05 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871692+0000", 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1714000000000002e-05 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896004+0000", 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024312099 2024-09-18T00:58:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896099+0000", 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5036000000000003e-05 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896270+0000", 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017098300000000001 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.465446568000001, 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:58:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15723 3.f 3:f7b50ba2:::rbd_data.117c53e529c1.0000000000000058:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.307177641999999, 2024-09-18T00:58:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026056645999999999, 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15723 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222235+0000", 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222242+0000", 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4710000000000004e-06 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222255+0000", 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2877999999999999e-05 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222261+0000", 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.489e-06 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222328+0000", 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7280999999999995e-05 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222502+0000", 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000173609 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.227120+0000", 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046180129999999998 2024-09-18T00:58:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248104+0000", 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020983979 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248166+0000", 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2340000000000003e-05 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248236+0000", 2024-09-18T00:58:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9227000000000007e-05 2024-09-18T00:58:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248291+0000", 2024-09-18T00:58:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5019000000000003e-05 2024-09-18T00:58:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:06.624 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:06.626 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:06.626 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:06.622+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:06.662 INFO:teuthology.orchestra.run.smithi042.stdout:1331 trunc from 0xa70a820 to 0x4abfaa2 2024-09-18T00:58:06.690 INFO:teuthology.orchestra.run.smithi042.stdout:1332 trunc from 0x4abfaa2 to 0x3527494 2024-09-18T00:58:06.716 INFO:teuthology.orchestra.run.smithi042.stdout:1333 read 0xf948ae thru 0xfa1aa5 (0xd1f8 bytes) 2024-09-18T00:58:06.726 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:06.722+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:06.739 INFO:teuthology.orchestra.run.smithi042.stdout:1334 write 0x7bf3189 thru 0x7c02a47 (0xf8bf bytes) 2024-09-18T00:58:06.746 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:06.749 INFO:teuthology.orchestra.run.smithi042.stdout:1336 trunc from 0x7c02a48 to 0x977e96e 2024-09-18T00:58:06.767 INFO:teuthology.orchestra.run.smithi042.stdout:1338 trunc from 0x977e96e to 0x456fcd9 2024-09-18T00:58:06.788 INFO:teuthology.orchestra.run.smithi042.stdout:1339 punch from 0x42c63a9 to 0x42d2100, (0xbd57 bytes) 2024-09-18T00:58:06.788 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:06.788 INFO:teuthology.orchestra.run.smithi042.stdout:1341 writesame 0x2b7f5f1 thru 0x2b852fc data_size 0x1e(0x5d0c bytes) 2024-09-18T00:58:06.793 INFO:teuthology.orchestra.run.smithi042.stdout:1343 read 0x446c876 thru 0x446f034 (0x27bf bytes) 2024-09-18T00:58:06.793 INFO:teuthology.orchestra.run.smithi042.stdout:1344 writesame 0x1060d1a thru 0x106f80c data_size 0x29(0xeaf3 bytes) 2024-09-18T00:58:06.801 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:06.801 INFO:teuthology.orchestra.run.smithi042.stdout:1347 read 0xb6e1dd thru 0xb70b8e (0x29b2 bytes) 2024-09-18T00:58:06.802 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:06.802 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:06.803 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:06.803 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:06.803 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:06.823 INFO:teuthology.orchestra.run.smithi042.stdout:1350 trunc from 0x456fcd9 to 0xdf186c0 2024-09-18T00:58:06.827 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:06.822+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:06.829 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:06.836 INFO:teuthology.orchestra.run.smithi042.stdout:1352 read 0xb8fe60f thru 0xb904802 (0x61f4 bytes) 2024-09-18T00:58:06.836 INFO:teuthology.orchestra.run.smithi042.stdout:1354 writesame 0x572f3c4 thru 0x573d426 data_size 0x1f(0xe063 bytes) 2024-09-18T00:58:06.842 INFO:teuthology.orchestra.run.smithi042.stdout:1357 read 0x8a9d16e thru 0x8a9ed21 (0x1bb4 bytes) 2024-09-18T00:58:06.843 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:06.843 INFO:teuthology.orchestra.run.smithi042.stdout:1359 read 0xd51b294 thru 0xd52a971 (0xf6de bytes) 2024-09-18T00:58:06.843 INFO:teuthology.orchestra.run.smithi042.stdout:1360 write 0x72a2287 thru 0x72ad0c7 (0xae41 bytes) 2024-09-18T00:58:06.848 INFO:teuthology.orchestra.run.smithi042.stdout:1361 read 0x791b9c4 thru 0x7925165 (0x97a2 bytes) 2024-09-18T00:58:06.848 INFO:teuthology.orchestra.run.smithi042.stdout:1363 write 0x7cc22ea thru 0x7cc4cb6 (0x29cd bytes) 2024-09-18T00:58:06.853 INFO:teuthology.orchestra.run.smithi042.stdout:1365 write 0x7ca5e34 thru 0x7cad7ad (0x797a bytes) 2024-09-18T00:58:06.855 INFO:teuthology.orchestra.run.smithi042.stdout:1366 punch from 0x2317eac to 0x23256c1, (0xd815 bytes) 2024-09-18T00:58:06.855 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:06.855 INFO:teuthology.orchestra.run.smithi042.stdout:1369 read 0x1106334 thru 0x11072e9 (0xfb6 bytes) 2024-09-18T00:58:06.855 INFO:teuthology.orchestra.run.smithi042.stdout:1370 write 0x52c8bc0 thru 0x52d807f (0xf4c0 bytes) 2024-09-18T00:58:06.861 INFO:teuthology.orchestra.run.smithi042.stdout:1371 read 0x7d13cf3 thru 0x7d1cab5 (0x8dc3 bytes) 2024-09-18T00:58:06.861 INFO:teuthology.orchestra.run.smithi042.stdout:1372 writesame 0xd888153 thru 0xd896ebf data_size 0x13(0xed6d bytes) 2024-09-18T00:58:06.868 INFO:teuthology.orchestra.run.smithi042.stdout:1373 writesame 0x93f75a9 thru 0x93fcbc8 data_size 0x10(0x5620 bytes) 2024-09-18T00:58:06.890 INFO:teuthology.orchestra.run.smithi042.stdout:1374 trunc from 0xdf186c0 to 0x44dca5e 2024-09-18T00:58:06.914 INFO:teuthology.orchestra.run.smithi042.stdout:1376 trunc from 0x44dca5e to 0x6aa170d 2024-09-18T00:58:06.926 INFO:teuthology.orchestra.run.smithi042.stdout:1377 read 0x2dd181f thru 0x2ddd7fa (0xbfdc bytes) 2024-09-18T00:58:06.926 INFO:teuthology.orchestra.run.smithi042.stdout:1380 write 0x13987f8 thru 0x139fb0d (0x7316 bytes) 2024-09-18T00:58:06.928 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:06.922+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:06.959 INFO:teuthology.orchestra.run.smithi042.stdout:1381 write 0xd3b9c41 thru 0xd3c034a (0x670a bytes) 2024-09-18T00:58:06.965 INFO:teuthology.orchestra.run.smithi042.stdout:1382 read 0x84c2aee thru 0x84c9f95 (0x74a8 bytes) 2024-09-18T00:58:06.965 INFO:teuthology.orchestra.run.smithi042.stdout:1383 write 0xc1dc605 thru 0xc1e68fc (0xa2f8 bytes) 2024-09-18T00:58:06.972 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:06.972 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:06.972 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:06.972 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:06.972 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:06.973 INFO:teuthology.orchestra.run.smithi042.stdout:1384 read 0x6503682 thru 0x650df26 (0xa8a5 bytes) 2024-09-18T00:58:06.973 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:06.973 INFO:teuthology.orchestra.run.smithi042.stdout:1388 write 0x4d99c3a thru 0x4da521d (0xb5e4 bytes) 2024-09-18T00:58:06.977 INFO:teuthology.orchestra.run.smithi042.stdout:1389 punch from 0x5cf49c3 to 0x5cf98c1, (0x4efe bytes) 2024-09-18T00:58:06.978 INFO:teuthology.orchestra.run.smithi042.stdout:1390 writesame 0x4f02679 thru 0x4f0f9b7 data_size 0x1(0xd33f bytes) 2024-09-18T00:58:06.986 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:07.030 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:07.026+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:07.040 INFO:teuthology.orchestra.run.smithi042.stdout:1391 read 0x2cfc2c2 thru 0x2d0129d (0x4fdc bytes) 2024-09-18T00:58:07.051 INFO:teuthology.orchestra.run.smithi042.stdout:1392 trunc from 0xd3c034b to 0x80628d9 2024-09-18T00:58:07.065 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T00:58:07.080 INFO:teuthology.orchestra.run.smithi042.stdout:1393 punch from 0x7e17564 to 0x7e203c0, (0x8e5c bytes) 2024-09-18T00:58:07.080 INFO:teuthology.orchestra.run.smithi042.stdout:1394 write 0x49f5007 thru 0x49f84e0 (0x34da bytes) 2024-09-18T00:58:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:07.086 INFO:teuthology.orchestra.run.smithi042.stdout:1398 read 0x7abb0d7 thru 0x7ac5d5f (0xac89 bytes) 2024-09-18T00:58:07.090 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T00:58:07.099 INFO:teuthology.orchestra.run.smithi042.stdout:1399 write 0x97d94e1 thru 0x97e8f43 (0xfa63 bytes) 2024-09-18T00:58:07.103 INFO:teuthology.orchestra.run.smithi042.stdout:1401 read 0x83abe9a thru 0x83ace41 (0xfa8 bytes) 2024-09-18T00:58:07.114 INFO:teuthology.orchestra.run.smithi042.stdout:1402 writesame 0xa0c63ff thru 0xa0c859b data_size 0x1(0x219d bytes) 2024-09-18T00:58:07.129 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:07.126+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:07.131 INFO:teuthology.orchestra.run.smithi042.stdout:1403 writesame 0xe0dc500 thru 0xe0e4bb1 data_size 0x15(0x86b2 bytes) 2024-09-18T00:58:07.131 INFO:teuthology.orchestra.run.smithi042.stdout:1404 write 0xfb43e0 thru 0xfb9ad3 (0x56f4 bytes) 2024-09-18T00:58:07.138 INFO:teuthology.orchestra.run.smithi042.stdout:1405 writesame 0xcf99123 thru 0xcf9aaaa data_size 0x1(0x1988 bytes) 2024-09-18T00:58:07.146 INFO:teuthology.orchestra.run.smithi042.stdout:1406 read 0xb223f3e thru 0xb226e56 (0x2f19 bytes) 2024-09-18T00:58:07.157 INFO:teuthology.orchestra.run.smithi042.stdout:1408 writesame 0xe75dba6 thru 0xe76b241 data_size 0x14(0xd69c bytes) 2024-09-18T00:58:07.157 INFO:teuthology.orchestra.run.smithi042.stdout:1409 writesame 0x54d8e80 thru 0x54e36de data_size 0x1(0xa85f bytes) 2024-09-18T00:58:07.173 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:07.173 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:07.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:07.173 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:07.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.173 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:58:07.173 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.881976283999997, 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:58:07.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:58:07.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:07.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:58:07.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:58:07.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:58:07.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:58:07.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7423 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.685677733999999, 2024-09-18T00:58:07.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045296661000000002, 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7423 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485621+0000", 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485627+0000", 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.156e-06 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485633+0000", 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789999999999999e-06 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530788+0000", 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045155001 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530853+0000", 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5634000000000002e-05 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530915+0000", 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2022999999999999e-05 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.184 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7424 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1146880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.685656743999999, 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045419355000000002, 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7424 2024-09-18T00:58:07.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485642+0000", 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0640000000000001e-06 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485649+0000", 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485654+0000", 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2329999999999997e-06 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530941+0000", 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045286602000000002 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531004+0000", 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3455999999999997e-05 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531059+0000", 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4771000000000001e-05 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.685635230999999, 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:58:07.189 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:58:07.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.685613793999998, 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.194 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:58:07.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.685518467000001, 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.198 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.685493028, 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.685470505000001, 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.685447064999998, 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.685423255, 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.68540089, 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.685374273000001, 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.681222502000001, 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.681201759, 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.681156194, 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.681129853000002, 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.681107183000002, 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.681088809999999, 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.681069021999999, 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.681042152, 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout:1413 punch from 0xbda47f6 to 0xbda96fa, (0x4f04 bytes) 2024-09-18T00:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout:1415 punch from 0x4dda6a5 to 0x4de0606, (0x5f61 bytes) 2024-09-18T00:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout:1416 writesame 0xe3007ee thru 0xe30f30c data_size 0x1(0xeb1f bytes) 2024-09-18T00:58:07.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout:1419 write 0x4c01253 thru 0x4c0158b (0x339 bytes) 2024-09-18T00:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout:1424 write 0xdf52862 thru 0xdf54f1f (0x26be bytes) 2024-09-18T00:58:07.268 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:07.262+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout:1425 read 0x12886c9 thru 0x1288763 (0x9b bytes) 2024-09-18T00:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout:1426 punch from 0x1be3456 to 0x1bedf80, (0xab2a bytes) 2024-09-18T00:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout:1427 read 0xd484fc8 thru 0xd48fece (0xaf07 bytes) 2024-09-18T00:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout:1428 punch from 0x2cfc986 to 0x2d03f7c, (0x75f6 bytes) 2024-09-18T00:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout:1429 write 0x22e6a70 thru 0x22f0b73 (0xa104 bytes) 2024-09-18T00:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout:1430 write 0x79e0510 thru 0x79ebb53 (0xb644 bytes) 2024-09-18T00:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout:1431 trunc from 0xe76b242 to 0x9e3f432 2024-09-18T00:58:07.312 INFO:teuthology.orchestra.run.smithi042.stdout:1433 trunc from 0x9e3f432 to 0x6e30804 2024-09-18T00:58:07.330 INFO:teuthology.orchestra.run.smithi042.stdout:1434 write 0x119a26e thru 0x119c742 (0x24d5 bytes) 2024-09-18T00:58:07.355 INFO:teuthology.orchestra.run.smithi042.stdout:1436 write 0xea17ece thru 0xea1ec8f (0x6dc2 bytes) 2024-09-18T00:58:07.360 INFO:teuthology.orchestra.run.smithi042.stdout:1437 read 0x93ea48 thru 0x942fd0 (0x4589 bytes) 2024-09-18T00:58:07.362 INFO:teuthology.orchestra.run.smithi042.stdout:1440 writesame 0x30f464e thru 0x3102360 data_size 0x93(0xdd13 bytes) 2024-09-18T00:58:07.366 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:07.362+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:07.369 INFO:teuthology.orchestra.run.smithi042.stdout:1441 punch from 0xbb0600f to 0xbb07b87, (0x1b78 bytes) 2024-09-18T00:58:07.369 INFO:teuthology.orchestra.run.smithi042.stdout:1442 writesame 0x72d719b thru 0x72e1b63 data_size 0x1(0xa9c9 bytes) 2024-09-18T00:58:07.401 INFO:teuthology.orchestra.run.smithi042.stdout:1443 read 0x2464984 thru 0x246c2f7 (0x7974 bytes) 2024-09-18T00:58:07.401 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:07.401 INFO:teuthology.orchestra.run.smithi042.stdout:1445 read 0x2f5d3bb thru 0x2f62ed8 (0x5b1e bytes) 2024-09-18T00:58:07.402 INFO:teuthology.orchestra.run.smithi042.stdout:1446 punch from 0x19c7452 to 0x19c96d5, (0x2283 bytes) 2024-09-18T00:58:07.403 INFO:teuthology.orchestra.run.smithi042.stdout:1448 writesame 0x5ab1295 thru 0x5ab17fa data_size 0x1(0x566 bytes) 2024-09-18T00:58:07.409 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:07.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:07.409 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:07.409 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:07.413 INFO:teuthology.orchestra.run.smithi042.stdout:1449 write 0x20fe1f4 thru 0x2102935 (0x4742 bytes) 2024-09-18T00:58:07.422 INFO:teuthology.orchestra.run.smithi042.stdout:1450 clone 15 order 25 su 4194304 sc 6 2024-09-18T00:58:07.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:07.468 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:07.462+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:07.568 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:07.566+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:07.584 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:07.585 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:07.585 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:07.598 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:07.669 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:07.662+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:07.724 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:07.724 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:07.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:07.724 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:07.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.724 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.724 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:07.724 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.644993134, 2024-09-18T00:58:07.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:58:07.724 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:58:07.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.644063543000001, 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:58:07.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:58:07.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:58:07.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:58:07.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:58:07.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:07.733 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.644049483, 2024-09-18T00:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.642758908000001, 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.642738435999998, 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.639943982999998, 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.639928799, 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.639905852999998, 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.635728214, 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.635706997, 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.635685614, 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.63178576, 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.631771559000001, 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.631753522, 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.631739455999998, 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.627714104999999, 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:07.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.627700093000001, 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.627686129000001, 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.865828175000001, 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.740351804, 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:07.819 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:07.820 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:07.814+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:07.920 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:07.914+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:07.994 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:07.994 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:07.994 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:07.994 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:08.008 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:08.050 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:08.046+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:08.077 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T00:58:08.091 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:58:08.103 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T00:58:08.121 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:08.118+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:08.185 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:08.185 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:08.185 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:08.186 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:08.186 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:08.204 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:08.246 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:08.242+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:08.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:08.322 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:08.318+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:08.373 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:08.373 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:08.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:08.373 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:08.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.373 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:58:08.373 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:08.373 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.055625749000001, 2024-09-18T00:58:08.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101264911, 2024-09-18T00:58:08.373 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.373 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4226", 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2155402750", 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313760+0000", 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.612e-06 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313768+0000", 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9249999999999995e-06 2024-09-18T00:58:08.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313786+0000", 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8253999999999998e-05 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313803+0000", 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313935+0000", 2024-09-18T00:58:08.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013180399999999999 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313954+0000", 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8768999999999999e-05 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.411874+0000", 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.097920646 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412292+0000", 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041777699999999998 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412675+0000", 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038262300000000001 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414019+0000", 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013436699999999999 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414872+0000", 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085320999999999997 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414955+0000", 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3405999999999994e-05 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.415023+0000", 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7825999999999997e-05 2024-09-18T00:58:08.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3739 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867528+0000", 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.501855604999999, 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020284267000000002, 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3739 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867528+0000", 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867528+0000", 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867529+0000", 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6400000000000002e-07 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867531+0000", 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867539+0000", 2024-09-18T00:58:08.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1520000000000003e-06 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867555+0000", 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6566999999999999e-05 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887588+0000", 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020032395000000001 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887691+0000", 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103582 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887812+0000", 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121344 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3740 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:08.384 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.501821023000002, 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020503747999999999, 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3740 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867566+0000", 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3159999999999999e-06 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867571+0000", 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9240000000000004e-06 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867576+0000", 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8210000000000001e-06 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887847+0000", 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020270468 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887952+0000", 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010541599999999999 2024-09-18T00:58:08.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888067+0000", 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000114391 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3741 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.501801144000002, 2024-09-18T00:58:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020744149999999999, 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3741 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0800000000000001e-07 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867585+0000", 2024-09-18T00:58:08.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867591+0000", 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3269999999999997e-06 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867597+0000", 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0750000000000003e-06 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888108+0000", 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020510589999999999 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888212+0000", 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104252 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888327+0000", 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011461800000000001 2024-09-18T00:58:08.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3742 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.501779613, 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020983208999999999, 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3742 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867605+0000", 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867613+0000", 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867620+0000", 2024-09-18T00:58:08.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.793e-06 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867625+0000", 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2619999999999998e-06 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888362+0000", 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020736747999999999 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888469+0000", 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106969 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888587+0000", 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118904 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3743 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:08.398 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.501752019000001, 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021178947, 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3743 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867634+0000", 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.956e-06 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867640+0000", 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0769999999999996e-06 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867645+0000", 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5619999999999997e-06 2024-09-18T00:58:08.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888622+0000", 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020977554999999998 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888715+0000", 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2905000000000006e-05 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888811+0000", 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5513999999999994e-05 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3744 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.501732740000001, 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021391389, 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.403 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3744 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8299999999999998e-07 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867653+0000", 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867659+0000", 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5999999999999997e-06 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867664+0000", 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8910000000000003e-06 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888846+0000", 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021181867 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888951+0000", 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105837 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889042+0000", 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1012000000000002e-05 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.407 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3745 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.501711487000001, 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021600477999999999, 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3745 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867673+0000", 2024-09-18T00:58:08.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2599999999999998e-07 2024-09-18T00:58:08.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867675+0000", 2024-09-18T00:58:08.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0499999999999999e-06 2024-09-18T00:58:08.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867682+0000", 2024-09-18T00:58:08.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0140000000000001e-06 2024-09-18T00:58:08.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867687+0000", 2024-09-18T00:58:08.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9790000000000001e-06 2024-09-18T00:58:08.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889079+0000", 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021392347999999999 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889182+0000", 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102712 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889273+0000", 2024-09-18T00:58:08.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1049000000000002e-05 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3746 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.501689899999999, 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021835075999999998, 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.414 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3746 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6100000000000002e-07 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867701+0000", 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867709+0000", 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7999999999999999e-06 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867715+0000", 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.699e-06 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889307+0000", 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021592239999999999 2024-09-18T00:58:08.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889410+0000", 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103093 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889529+0000", 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011872299999999999 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3747 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.501659751999998, 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022084972000000001, 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3747 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.419 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867725+0000", 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5199999999999996e-07 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867735+0000", 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7790000000000007e-06 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867743+0000", 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1389999999999995e-06 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867748+0000", 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7209999999999999e-06 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889588+0000", 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021839447000000001 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889706+0000", 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117748 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889809+0000", 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103486 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3748 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.501624573000001, 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022206404999999999, 2024-09-18T00:58:08.423 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3748 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:08.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867760+0000", 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-07 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867762+0000", 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6079999999999998e-06 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867769+0000", 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9840000000000004e-06 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867785+0000", 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5616000000000001e-05 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889845+0000", 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022060283 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889913+0000", 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8104999999999999e-05 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889966+0000", 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2340999999999999e-05 2024-09-18T00:58:08.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.428 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.428 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.428 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7282 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:08.428 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473428+0000", 2024-09-18T00:58:08.428 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.895955661999999, 2024-09-18T00:58:08.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020338440999999999, 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7282 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473428+0000", 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473428+0000", 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473429+0000", 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.23e-07 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473434+0000", 2024-09-18T00:58:08.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7740000000000004e-06 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473449+0000", 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4922e-05 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473481+0000", 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2382999999999998e-05 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493569+0000", 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020087588 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493640+0000", 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1237999999999998e-05 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493767+0000", 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126713 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.432 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7283 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1703936~2106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.895883361999999, 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02059619, 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7283 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473501+0000", 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473507+0000", 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2939999999999996e-06 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473522+0000", 2024-09-18T00:58:08.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5386000000000001e-05 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473534+0000", 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1917e-05 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493796+0000", 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020262184999999999 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493907+0000", 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011105800000000001 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494097+0000", 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018939299999999999 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.437 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7284 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.895833422999999, 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020722913999999999, 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7284 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473551+0000", 2024-09-18T00:58:08.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999999e-07 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473556+0000", 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8949999999999997e-06 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473571+0000", 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.486e-05 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473603+0000", 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2375000000000001e-05 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494129+0000", 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020525244000000002 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494206+0000", 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7601000000000005e-05 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494273+0000", 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7101999999999996e-05 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7292 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.895763152000001, 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020882831000000001, 2024-09-18T00:58:08.442 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.443 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7292 2024-09-18T00:58:08.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.443 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473622+0000", 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.78e-07 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473626+0000", 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7330000000000003e-06 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473640+0000", 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4034e-05 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473651+0000", 2024-09-18T00:58:08.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0937999999999999e-05 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494300+0000", 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020649238 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494392+0000", 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1184e-05 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494504+0000", 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111826 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7293 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.895717834999999, 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021019589000000002, 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:08.447 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7293 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:08.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473667+0000", 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6600000000000005e-07 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473682+0000", 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.483e-05 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.477101+0000", 2024-09-18T00:58:08.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034192710000000002 2024-09-18T00:58:08.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.477120+0000", 2024-09-18T00:58:08.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9202999999999999e-05 2024-09-18T00:58:08.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494533+0000", 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017412838999999999 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494603+0000", 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9947e-05 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494686+0000", 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2633000000000007e-05 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15500 3.11 3:8ed6a97b:::rbd_data.117c53e529c1.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.554865728999999, 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046818956000000002, 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15500 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814521+0000", 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.977e-06 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814536+0000", 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.487e-05 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814572+0000", 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5567e-05 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814588+0000", 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6546e-05 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814675+0000", 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6675999999999995e-05 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814839+0000", 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016414199999999999 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.819908+0000", 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050695280000000002 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861139+0000", 2024-09-18T00:58:08.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041230771999999999 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861208+0000", 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8722999999999999e-05 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861277+0000", 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9246999999999996e-05 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861337+0000", 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9908000000000003e-05 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:21989 3.1d 3:beac871e:::rbd_object_map.117c21a2e20a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8399990930000003, 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021103017000000002, 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 21989 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2399999999999999e-07 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529389+0000", 2024-09-18T00:58:08.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3759999999999999e-06 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529398+0000", 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4080000000000004e-06 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529404+0000", 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8919999999999997e-06 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.544686+0000", 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015282662000000001 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.544799+0000", 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011215800000000001 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.545254+0000", 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045577600000000002 2024-09-18T00:58:08.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.547172+0000", 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019178999999999999 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550317+0000", 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031449310000000001 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550437+0000", 2024-09-18T00:58:08.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119963 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550488+0000", 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0526999999999999e-05 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:21990 3.1d 3:beac871e:::rbd_object_map.117c21a2e20a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:08.464 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.8399706460000003, 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023815820000000001, 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 21990 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:08.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529414+0000", 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529417+0000", 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2849999999999999e-06 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529425+0000", 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7740000000000004e-06 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529431+0000", 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8939999999999998e-06 2024-09-18T00:58:08.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546090+0000", 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016659706 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546185+0000", 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4889999999999994e-05 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546569+0000", 2024-09-18T00:58:08.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038352599999999997 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.547692+0000", 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001123024 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553127+0000", 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054353630000000003 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553188+0000", 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1247999999999994e-05 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553229+0000", 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0646999999999999e-05 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.470 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26860 3.a 3:523ff797:::rbd_data.117c4c7db1a0.0000000000000039:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.99669160999999995, 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028475341000000001, 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26860 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372693+0000", 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0550000000000001e-06 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372699+0000", 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372714+0000", 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4673e-05 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372722+0000", 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4659999999999999e-06 2024-09-18T00:58:08.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372787+0000", 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5288000000000002e-05 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372874+0000", 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6485999999999998e-05 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:08.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.377992+0000", 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051177159999999996 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401008+0000", 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02301603 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401026+0000", 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712e-05 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401082+0000", 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5648e-05 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401168+0000", 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5476999999999995e-05 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:08.476 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:08.477 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:08.478 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:08.478 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone14 from 0xea1ec90 (overlap 0x3527494) to 0x0 2024-09-18T00:58:08.479 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:08.474+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:08.483 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:08.483 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:08.501 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:08.501 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:08.518 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:08.518 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:08.535 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:08.535 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:08.555 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:08.555 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:08.568 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-18T00:58:08.575 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:08.575 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:08.579 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:08.574+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:08.613 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:08.614 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:08.626 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:08.626 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:08.638 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:08.639 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:08.641 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:1451 write 0x2c99077 thru 0x2c99ba0 (0xb2a bytes) 2024-09-18T00:58:08.650 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:08.650 INFO:teuthology.orchestra.run.smithi042.stdout:1453 read 0x26fb051 thru 0x26feaba (0x3a6a bytes) 2024-09-18T00:58:08.650 INFO:teuthology.orchestra.run.smithi042.stdout:1454 writesame 0x5197723 thru 0x519f425 data_size 0x1(0x7d03 bytes) 2024-09-18T00:58:08.685 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:08.682+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:08.723 INFO:teuthology.orchestra.run.smithi042.stdout:1455 trunc from 0xea1ec90 to 0x523989 2024-09-18T00:58:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:1456 punch from 0x4a5b25 to 0x4b2761, (0xcc3c bytes) 2024-09-18T00:58:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:1457 punch from 0x4e415 to 0x4ff2c, (0x1b17 bytes) 2024-09-18T00:58:08.781 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:08.774+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:08.813 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:08.813 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:08.813 INFO:teuthology.orchestra.run.smithi042.stdout:1461 trunc from 0x523989 to 0xeb883a0 2024-09-18T00:58:08.828 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:08.829 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:08.830 INFO:teuthology.orchestra.run.smithi042.stdout:1464 read 0xd0e6de0 thru 0xd0e6f1d (0x13e bytes) 2024-09-18T00:58:08.830 INFO:teuthology.orchestra.run.smithi042.stdout:1465 punch from 0x5134fe7 to 0x513670a, (0x1723 bytes) 2024-09-18T00:58:08.830 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout:1467 trunc from 0xeb883a0 to 0x5c1bcdc 2024-09-18T00:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout:1468 read 0x38a6361 thru 0x38a7241 (0xee1 bytes) 2024-09-18T00:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout:1469 punch from 0x169dc1f to 0x16a72bc, (0x969d bytes) 2024-09-18T00:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout:1470 write 0x233b728 thru 0x2341691 (0x5f6a bytes) 2024-09-18T00:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout:1473 write 0x5ae85fe thru 0x5af3a76 (0xb479 bytes) 2024-09-18T00:58:08.882 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:08.878+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout:1474 writesame 0x977fa51 thru 0x9783e59 data_size 0x1(0x4409 bytes) 2024-09-18T00:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout:1475 writesame 0xb83cdee thru 0xb84a598 data_size 0x1f(0xd7ab bytes) 2024-09-18T00:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout:1476 write 0x56bd8af thru 0x56c4e8a (0x75dc bytes) 2024-09-18T00:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout:1477 punch from 0x7bfb202 to 0x7bfe001, (0x2dff bytes) 2024-09-18T00:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout:1478 read 0x29b902c thru 0x29c6500 (0xd4d5 bytes) 2024-09-18T00:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xee4a7d0 2024-09-18T00:58:08.937 INFO:teuthology.orchestra.run.smithi042.stdout:1479 trunc from 0xb84a599 to 0xee4a7d0 2024-09-18T00:58:08.950 INFO:teuthology.orchestra.run.smithi042.stdout:1481 writesame 0x7df9810 thru 0x7e02bef data_size 0x5b(0x93e0 bytes) 2024-09-18T00:58:08.960 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:08.960 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:08.965 INFO:teuthology.orchestra.run.smithi042.stdout:1482 trunc from 0xee4a7d0 to 0xa195917 2024-09-18T00:58:08.976 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:08.983 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:08.978+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:09.000 INFO:teuthology.orchestra.run.smithi042.stdout:1483 writesame 0xb1f5eca thru 0xb1fae9e data_size 0x1(0x4fd5 bytes) 2024-09-18T00:58:09.003 INFO:teuthology.orchestra.run.smithi042.stdout:1484 writesame 0x84da589 thru 0x84dd5cd data_size 0x3(0x3045 bytes) 2024-09-18T00:58:09.018 INFO:teuthology.orchestra.run.smithi042.stdout:1485 trunc from 0xb1fae9f to 0xedd23dd 2024-09-18T00:58:09.031 INFO:teuthology.orchestra.run.smithi042.stdout:1486 write 0x8679315 thru 0x867eab8 (0x57a4 bytes) 2024-09-18T00:58:09.035 INFO:teuthology.orchestra.run.smithi042.stdout:1487 writesame 0xcb6f0cd thru 0xcb6fabd data_size 0x1(0x9f1 bytes) 2024-09-18T00:58:09.043 INFO:teuthology.orchestra.run.smithi042.stdout:1488 write 0x87eda00 thru 0x87fd958 (0xff59 bytes) 2024-09-18T00:58:09.047 INFO:teuthology.orchestra.run.smithi042.stdout:1491 read 0xb3f81f4 thru 0xb3ffc5d (0x7a6a bytes) 2024-09-18T00:58:09.047 INFO:teuthology.orchestra.run.smithi042.stdout:1492 read 0x2c10f36 thru 0x2c1ca0b (0xbad6 bytes) 2024-09-18T00:58:09.048 INFO:teuthology.orchestra.run.smithi042.stdout:1493 writesame 0xb1de2b3 thru 0xb1dfd04 data_size 0x3(0x1a52 bytes) 2024-09-18T00:58:09.052 INFO:teuthology.orchestra.run.smithi042.stdout:1494 read 0x2c0bac5 thru 0x2c19bc2 (0xe0fe bytes) 2024-09-18T00:58:09.052 INFO:teuthology.orchestra.run.smithi042.stdout:1495 writesame 0xb53f908 thru 0xb5444f8 data_size 0x1(0x4bf1 bytes) 2024-09-18T00:58:09.068 INFO:teuthology.orchestra.run.smithi042.stdout:1496 read 0xdf1c5c0 thru 0xdf1d6d8 (0x1119 bytes) 2024-09-18T00:58:09.068 INFO:teuthology.orchestra.run.smithi042.stdout:1497 write 0x72e74c2 thru 0x72ebce5 (0x4824 bytes) 2024-09-18T00:58:09.071 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.071 INFO:teuthology.orchestra.run.smithi042.stdout:1499 read 0x8c7c6ee thru 0x8c7db9f (0x14b2 bytes) 2024-09-18T00:58:09.072 INFO:teuthology.orchestra.run.smithi042.stdout:1500 writesame 0xe94ccff thru 0xe95c996 data_size 0x89(0xfc98 bytes) 2024-09-18T00:58:09.079 INFO:teuthology.orchestra.run.smithi042.stdout:1502 write 0x5e8e8a thru 0x5ede25 (0x4f9c bytes) 2024-09-18T00:58:09.083 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:58:09.084 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:09.078+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:09.089 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T00:58:09.099 INFO:teuthology.orchestra.run.smithi042.stdout:1503 trunc from 0xedd23dd to 0x8d995d9 2024-09-18T00:58:09.115 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T00:58:09.151 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:09.152 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:09.167 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:09.178 INFO:teuthology.orchestra.run.smithi042.stdout:1504 write 0x7ae2de1 thru 0x7aeb1e7 (0x8407 bytes) 2024-09-18T00:58:09.181 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.181 INFO:teuthology.orchestra.run.smithi042.stdout:1506 punch from 0x2fcd948 to 0x2fda8cb, (0xcf83 bytes) 2024-09-18T00:58:09.181 INFO:teuthology.orchestra.run.smithi042.stdout:1510 read 0x82626c0 thru 0x826a193 (0x7ad4 bytes) 2024-09-18T00:58:09.196 INFO:teuthology.orchestra.run.smithi042.stdout:1511 write 0xbc6ba27 thru 0xbc6d106 (0x16e0 bytes) 2024-09-18T00:58:09.200 INFO:teuthology.orchestra.run.smithi042.stdout:1512 writesame 0x34a15c6 thru 0x34aa9da data_size 0x1(0x9415 bytes) 2024-09-18T00:58:09.213 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:09.210+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:09.235 INFO:teuthology.orchestra.run.smithi042.stdout:1513 write 0x8ab5f4b thru 0x8abc07d (0x6133 bytes) 2024-09-18T00:58:09.237 INFO:teuthology.orchestra.run.smithi042.stdout:1516 read 0xa7021b8 thru 0xa706bb4 (0x49fd bytes) 2024-09-18T00:58:09.238 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.238 INFO:teuthology.orchestra.run.smithi042.stdout:1518 read 0xe54d1f thru 0xe5f5b4 (0xa896 bytes) 2024-09-18T00:58:09.239 INFO:teuthology.orchestra.run.smithi042.stdout:1519 punch from 0x3a4d83e to 0x3a5d27f, (0xfa41 bytes) 2024-09-18T00:58:09.258 INFO:teuthology.orchestra.run.smithi042.stdout:1520 trunc from 0xbc6d107 to 0x33969d7 2024-09-18T00:58:09.278 INFO:teuthology.orchestra.run.smithi042.stdout:1521 read 0x1e5bb0f thru 0x1e64e42 (0x9334 bytes) 2024-09-18T00:58:09.285 INFO:teuthology.orchestra.run.smithi042.stdout:1522 trunc from 0x33969d7 to 0x64971b2 2024-09-18T00:58:09.286 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:09.282+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:09.289 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:09.289 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:09.298 INFO:teuthology.orchestra.run.smithi042.stdout:1524 read 0x489e334 thru 0x48a2fa3 (0x4c70 bytes) 2024-09-18T00:58:09.299 INFO:teuthology.orchestra.run.smithi042.stdout:1525 punch from 0x39aba9a to 0x39b424d, (0x87b3 bytes) 2024-09-18T00:58:09.299 INFO:teuthology.orchestra.run.smithi042.stdout:1526 writesame 0x69c9b8 thru 0x6a5910 data_size 0x1(0x8f59 bytes) 2024-09-18T00:58:09.304 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:09.334 INFO:teuthology.orchestra.run.smithi042.stdout:1527 read 0x17c61ea thru 0x17d32a5 (0xd0bc bytes) 2024-09-18T00:58:09.334 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.342 INFO:teuthology.orchestra.run.smithi042.stdout:1530 trunc from 0x64971b2 to 0xae88bb1 2024-09-18T00:58:09.355 INFO:teuthology.orchestra.run.smithi042.stdout:1532 trunc from 0xae88bb1 to 0x8133041 2024-09-18T00:58:09.375 INFO:teuthology.orchestra.run.smithi042.stdout:1533 trunc from 0x8133041 to 0x3469eff 2024-09-18T00:58:09.388 INFO:teuthology.orchestra.run.smithi042.stdout:1534 punch from 0x200e7f to 0x2057a5, (0x4926 bytes) 2024-09-18T00:58:09.388 INFO:teuthology.orchestra.run.smithi042.stdout:1535 writesame 0x2bcef3a thru 0x2bd8d9f data_size 0x19(0x9e66 bytes) 2024-09-18T00:58:09.393 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:09.390+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:09.423 INFO:teuthology.orchestra.run.smithi042.stdout:1536 write 0xe6d9538 thru 0xe6e676c (0xd235 bytes) 2024-09-18T00:58:09.429 INFO:teuthology.orchestra.run.smithi042.stdout:1537 writesame 0xe563531 thru 0xe5655b6 data_size 0x17(0x2086 bytes) 2024-09-18T00:58:09.446 INFO:teuthology.orchestra.run.smithi042.stdout:1538 trunc from 0xe6e676d to 0x8226ac5 2024-09-18T00:58:09.480 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:09.480 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:09.480 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.480 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.487 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:09.482+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:09.494 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:09.494 INFO:teuthology.orchestra.run.smithi042.stdout:1542 trunc from 0x8226ac5 to 0x2131eaf 2024-09-18T00:58:09.507 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.518 INFO:teuthology.orchestra.run.smithi042.stdout:1545 write 0x219d79b thru 0x21a5049 (0x78af bytes) 2024-09-18T00:58:09.520 INFO:teuthology.orchestra.run.smithi042.stdout:1547 read 0x15c732d thru 0x15cc18c (0x4e60 bytes) 2024-09-18T00:58:09.521 INFO:teuthology.orchestra.run.smithi042.stdout:1549 read 0x52829b thru 0x5347c9 (0xc52f bytes) 2024-09-18T00:58:09.565 INFO:teuthology.orchestra.run.smithi042.stdout:1550 write 0xe1dfae4 thru 0xe1e04b0 (0x9cd bytes) 2024-09-18T00:58:09.570 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.570 INFO:teuthology.orchestra.run.smithi042.stdout:1554 punch from 0x83d20f0 to 0x83de8db, (0xc7eb bytes) 2024-09-18T00:58:09.576 INFO:teuthology.orchestra.run.smithi042.stdout:1555 trunc from 0xe1e04b1 to 0xc04791a 2024-09-18T00:58:09.588 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:09.582+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:09.598 INFO:teuthology.orchestra.run.smithi042.stdout:1556 writesame 0xabf05b thru 0xaca866 data_size 0x2(0xb80c bytes) 2024-09-18T00:58:09.614 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.614 INFO:teuthology.orchestra.run.smithi042.stdout:1559 punch from 0xbf47b9 to 0xbfe488, (0x9ccf bytes) 2024-09-18T00:58:09.614 INFO:teuthology.orchestra.run.smithi042.stdout:1562 write 0x2e5fec8 thru 0x2e6c6c5 (0xc7fe bytes) 2024-09-18T00:58:09.637 INFO:teuthology.orchestra.run.smithi042.stdout:1563 trunc from 0xc04791a to 0x5019bcf 2024-09-18T00:58:09.660 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.662 INFO:teuthology.orchestra.run.smithi042.stdout:1565 trunc from 0x5019bcf to 0x5c5d0a9 2024-09-18T00:58:09.678 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:09.678 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:09.689 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:09.682+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:09.692 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:09.714 INFO:teuthology.orchestra.run.smithi042.stdout:1567 write 0xda0cd91 thru 0xda13a75 (0x6ce5 bytes) 2024-09-18T00:58:09.754 INFO:teuthology.orchestra.run.smithi042.stdout:1568 trunc from 0xda13a76 to 0x1461708 2024-09-18T00:58:09.783 INFO:teuthology.orchestra.run.smithi042.stdout:1569 read 0x88277f thru 0x8836ae (0xf30 bytes) 2024-09-18T00:58:09.789 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:09.782+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:09.818 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:09.819 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:09.826 INFO:teuthology.orchestra.run.smithi042.stdout:1570 write 0xc7875e8 thru 0xc793107 (0xbb20 bytes) 2024-09-18T00:58:09.833 INFO:teuthology.orchestra.run.smithi042.stdout:1571 write 0x5b7eeba thru 0x5b81616 (0x275d bytes) 2024-09-18T00:58:09.834 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.836 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.865 INFO:teuthology.orchestra.run.smithi042.stdout:1575 trunc from 0xc793108 to 0x1832c48 2024-09-18T00:58:09.890 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:09.886+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:09.891 INFO:teuthology.orchestra.run.smithi042.stdout:1576 read 0x11e2ac6 thru 0x11e5b3b (0x3076 bytes) 2024-09-18T00:58:09.922 INFO:teuthology.orchestra.run.smithi042.stdout:1578 trunc from 0x1832c48 to 0xd21362a 2024-09-18T00:58:09.934 INFO:teuthology.orchestra.run.smithi042.stdout:1579 write 0x82cc28b thru 0x82dbf2a (0xfca0 bytes) 2024-09-18T00:58:09.938 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.938 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.938 INFO:teuthology.orchestra.run.smithi042.stdout:1584 write 0x691ac5c thru 0x6928aa9 (0xde4e bytes) 2024-09-18T00:58:09.942 INFO:teuthology.orchestra.run.smithi042.stdout:1586 read 0x85116d8 thru 0x851ed13 (0xd63c bytes) 2024-09-18T00:58:09.943 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:09.950 INFO:teuthology.orchestra.run.smithi042.stdout:1590 trunc from 0xd21362a to 0xa8059d0 2024-09-18T00:58:09.963 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:09.964 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:1591 writesame 0x6edc659 thru 0x6ee61e7 data_size 0x1(0x9b8f bytes) 2024-09-18T00:58:09.979 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:09.990 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:09.986+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:10.041 INFO:teuthology.orchestra.run.smithi042.stdout:1594 trunc from 0xa8059d0 to 0x3db8d16 2024-09-18T00:58:10.066 INFO:teuthology.orchestra.run.smithi042.stdout:1595 trunc from 0x3db8d16 to 0x9890914 2024-09-18T00:58:10.076 INFO:teuthology.orchestra.run.smithi042.stdout:1596 punch from 0x19f04bd to 0x19f8862, (0x83a5 bytes) 2024-09-18T00:58:10.091 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:10.086+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:10.095 INFO:teuthology.orchestra.run.smithi042.stdout:1597 writesame 0xb1d413b thru 0xb1d8e76 data_size 0x2(0x4d3c bytes) 2024-09-18T00:58:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T00:58:10.121 INFO:teuthology.orchestra.run.smithi042.stdout:1598 trunc from 0xb1d8e77 to 0x23e175e 2024-09-18T00:58:10.134 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T00:58:10.134 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:58:10.141 INFO:teuthology.orchestra.run.smithi042.stdout:1599 read 0x100f494 thru 0x1019031 (0x9b9e bytes) 2024-09-18T00:58:10.141 INFO:teuthology.orchestra.run.smithi042.stdout:1600 read 0xdbaf21 thru 0xdc6e71 (0xbf51 bytes) 2024-09-18T00:58:10.142 INFO:teuthology.orchestra.run.smithi042.stdout:1601 read 0x1d03ec3 thru 0x1d11927 (0xda65 bytes) 2024-09-18T00:58:10.157 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:10.158 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:10.160 INFO:teuthology.orchestra.run.smithi042.stdout:1603 writesame 0x521bbb9 thru 0x521f15b data_size 0x17(0x35a3 bytes) 2024-09-18T00:58:10.162 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:10.172 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:10.180 INFO:teuthology.orchestra.run.smithi042.stdout:1605 writesame 0x9585f48 thru 0x95943a8 data_size 0xb(0xe461 bytes) 2024-09-18T00:58:10.182 INFO:teuthology.orchestra.run.smithi042.stdout:1607 write 0xc51eb thru 0xca9fa (0x5810 bytes) 2024-09-18T00:58:10.184 INFO:teuthology.orchestra.run.smithi042.stdout:1611 write 0x1aa7de7 thru 0x1ab7668 (0xf882 bytes) 2024-09-18T00:58:10.187 INFO:teuthology.orchestra.run.smithi042.stdout:1612 read 0x149277e thru 0x149f581 (0xce04 bytes) 2024-09-18T00:58:10.187 INFO:teuthology.orchestra.run.smithi042.stdout:1613 clone 16 order 18 su 2048 sc 6 2024-09-18T00:58:10.221 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:10.218+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:10.291 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:10.292 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:10.292 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:10.286+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:10.305 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:10.393 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:10.386+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:10.458 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:58:10.458 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:58:10.470 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:10.470 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:10.477 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-18T00:58:10.097340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":342827136,"num_objects":859,"num_object_clones":134,"num_object_copies":1718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":859,"num_whiteouts":430,"num_read":37301,"num_read_kb":254758,"num_write":5462,"num_write_kb":238582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4610,"ondisk_log_size":4610,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":524776,"kb_used_data":233680,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754449944,"statfs":{"total":773094113280,"available":772556742656,"internally_reserved":0,"allocated":239288320,"data_stored":249914854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131072,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":63,"num_read_kb":304,"num_write":8,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.037653"},"pg_stats":[{"pgid":"3.1f","version":"45'108","reported_seq":663,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.830008+0000","last_change":"2024-09-18T00:57:38.909849+0000","last_active":"2024-09-18T00:58:05.830008+0000","last_peered":"2024-09-18T00:58:05.830008+0000","last_clean":"2024-09-18T00:58:05.830008+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T00:58:05.830008+0000","last_undegraded":"2024-09-18T00:58:05.830008+0000","last_fullsized":"2024-09-18T00:58:05.830008+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:40:00.440095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001565786,"stat_sum":{"num_bytes":1474576,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":14,"num_read":982,"num_read_kb":2293,"num_write":92,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"45'83","reported_seq":826,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.694076+0000","last_change":"2024-09-18T00:57:38.908434+0000","last_active":"2024-09-18T00:58:05.694076+0000","last_peered":"2024-09-18T00:58:05.694076+0000","last_clean":"2024-09-18T00:58:05.694076+0000","last_became_active":"2024-09-18T00:57:36.897948+0000","last_became_peered":"2024-09-18T00:57:36.897948+0000","last_unstale":"2024-09-18T00:58:05.694076+0000","last_undegraded":"2024-09-18T00:58:05.694076+0000","last_fullsized":"2024-09-18T00:58:05.694076+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:42:05.883003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040574399999999998,"stat_sum":{"num_bytes":5035983,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":19,"num_read":720,"num_read_kb":4752,"num_write":53,"num_write_kb":303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"45'293","reported_seq":974,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.133005+0000","last_change":"2024-09-18T00:57:38.902294+0000","last_active":"2024-09-18T00:58:05.133005+0000","last_peered":"2024-09-18T00:58:05.133005+0000","last_clean":"2024-09-18T00:58:05.133005+0000","last_became_active":"2024-09-18T00:57:36.906790+0000","last_became_peered":"2024-09-18T00:57:36.906790+0000","last_unstale":"2024-09-18T00:58:05.133005+0000","last_undegraded":"2024-09-18T00:58:05.133005+0000","last_fullsized":"2024-09-18T00:58:05.133005+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:34:19.660233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039089,"stat_sum":{"num_bytes":8240308,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":1757,"num_read_kb":12004,"num_write":453,"num_write_kb":1708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"45'282","reported_seq":1226,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.240177+0000","last_change":"2024-09-18T00:57:38.901600+0000","last_active":"2024-09-18T00:58:05.240177+0000","last_peered":"2024-09-18T00:58:05.240177+0000","last_clean":"2024-09-18T00:58:05.240177+0000","last_became_active":"2024-09-18T00:57:36.906003+0000","last_became_peered":"2024-09-18T00:57:36.906003+0000","last_unstale":"2024-09-18T00:58:05.240177+0000","last_undegraded":"2024-09-18T00:58:05.240177+0000","last_fullsized":"2024-09-18T00:58:05.240177+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:47.263915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000711266,"stat_sum":{"num_bytes":2320457,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":2243,"num_read_kb":8362,"num_write":349,"num_write_kb":1522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"45'287","reported_seq":1733,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.641256+0000","last_change":"2024-09-18T00:57:38.921854+0000","last_active":"2024-09-18T00:58:05.641256+0000","last_peered":"2024-09-18T00:58:05.641256+0000","last_clean":"2024-09-18T00:58:05.641256+0000","last_became_active":"2024-09-18T00:57:36.913393+0000","last_became_peered":"2024-09-18T00:57:36.913393+0000","last_unstale":"2024-09-18T00:58:05.641256+0000","last_undegraded":"2024-09-18T00:58:05.641256+0000","last_fullsized":"2024-09-18T00:58:05.641256+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:49:14.675049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002554022,"stat_sum":{"num_bytes":14881739,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":2406,"num_read_kb":9573,"num_write":423,"num_write_kb":8944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"45'132","reported_seq":1286,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.799313+0000","last_change":"2024-09-18T00:57:38.901805+0000","last_active":"2024-09-18T00:58:05.799313+0000","last_peered":"2024-09-18T00:58:05.799313+0000","last_clean":"2024-09-18T00:58:05.799313+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T00:58:05.799313+0000","last_undegraded":"2024-09-18T00:58:05.799313+0000","last_fullsized":"2024-09-18T00:58:05.799313+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:09.880267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098070899999999996,"stat_sum":{"num_bytes":26226007,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":1854,"num_read_kb":7668,"num_write":108,"num_write_kb":605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"45'86","reported_seq":1797,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.400596+0000","last_change":"2024-09-18T00:57:38.922010+0000","last_active":"2024-09-18T00:58:05.400596+0000","last_peered":"2024-09-18T00:58:05.400596+0000","last_clean":"2024-09-18T00:58:05.400596+0000","last_became_active":"2024-09-18T00:57:36.910457+0000","last_became_peered":"2024-09-18T00:57:36.910457+0000","last_unstale":"2024-09-18T00:58:05.400596+0000","last_undegraded":"2024-09-18T00:58:05.400596+0000","last_fullsized":"2024-09-18T00:58:05.400596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:55.403260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026842849999999998,"stat_sum":{"num_bytes":14881824,"num_objects":29,"num_object_clones":3,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":1693,"num_read_kb":9073,"num_write":94,"num_write_kb":8711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"45'163","reported_seq":1016,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.644876+0000","last_change":"2024-09-18T00:57:38.908144+0000","last_active":"2024-09-18T00:58:05.644876+0000","last_peered":"2024-09-18T00:58:05.644876+0000","last_clean":"2024-09-18T00:58:05.644876+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T00:58:05.644876+0000","last_undegraded":"2024-09-18T00:58:05.644876+0000","last_fullsized":"2024-09-18T00:58:05.644876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:04.349457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000871597,"stat_sum":{"num_bytes":28436710,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":1428,"num_read_kb":1474,"num_write":175,"num_write_kb":33827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"45'106","reported_seq":649,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.606953+0000","last_change":"2024-09-18T00:57:38.910038+0000","last_active":"2024-09-18T00:58:05.606953+0000","last_peered":"2024-09-18T00:58:05.606953+0000","last_clean":"2024-09-18T00:58:05.606953+0000","last_became_active":"2024-09-18T00:57:36.895915+0000","last_became_peered":"2024-09-18T00:57:36.895915+0000","last_unstale":"2024-09-18T00:58:05.606953+0000","last_undegraded":"2024-09-18T00:58:05.606953+0000","last_fullsized":"2024-09-18T00:58:05.606953+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:23:48.528113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001269769,"stat_sum":{"num_bytes":7187217,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":805,"num_read_kb":11695,"num_write":93,"num_write_kb":4072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"45'166","reported_seq":2411,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.132073+0000","last_change":"2024-09-18T00:57:39.381845+0000","last_active":"2024-09-18T00:58:05.132073+0000","last_peered":"2024-09-18T00:58:05.132073+0000","last_clean":"2024-09-18T00:58:05.132073+0000","last_became_active":"2024-09-18T00:57:36.894538+0000","last_became_peered":"2024-09-18T00:57:36.894538+0000","last_unstale":"2024-09-18T00:58:05.132073+0000","last_undegraded":"2024-09-18T00:58:05.132073+0000","last_fullsized":"2024-09-18T00:58:05.132073+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:28:34.626070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076552,"stat_sum":{"num_bytes":7716198,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":2765,"num_read_kb":12240,"num_write":300,"num_write_kb":14789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"45'111","reported_seq":591,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.642020+0000","last_change":"2024-09-18T00:57:38.909647+0000","last_active":"2024-09-18T00:58:05.642020+0000","last_peered":"2024-09-18T00:58:05.642020+0000","last_clean":"2024-09-18T00:58:05.642020+0000","last_became_active":"2024-09-18T00:57:36.902977+0000","last_became_peered":"2024-09-18T00:57:36.902977+0000","last_unstale":"2024-09-18T00:58:05.642020+0000","last_undegraded":"2024-09-18T00:58:05.642020+0000","last_fullsized":"2024-09-18T00:58:05.642020+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:40:26.460414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051351900000000004,"stat_sum":{"num_bytes":23482430,"num_objects":29,"num_object_clones":3,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":17,"num_read":446,"num_read_kb":7407,"num_write":140,"num_write_kb":38140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"45'40","reported_seq":130,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.584297+0000","last_change":"2024-09-18T00:57:38.902578+0000","last_active":"2024-09-18T00:58:05.584297+0000","last_peered":"2024-09-18T00:58:05.584297+0000","last_clean":"2024-09-18T00:58:05.584297+0000","last_became_active":"2024-09-18T00:57:36.894977+0000","last_became_peered":"2024-09-18T00:57:36.894977+0000","last_unstale":"2024-09-18T00:58:05.584297+0000","last_undegraded":"2024-09-18T00:58:05.584297+0000","last_fullsized":"2024-09-18T00:58:05.584297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:33:26.607066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028223319999999999,"stat_sum":{"num_bytes":787963,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":6,"num_read":42,"num_read_kb":35,"num_write":33,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.053230+0000","last_change":"2024-09-18T00:57:29.328163+0000","last_active":"2024-09-18T00:58:05.053230+0000","last_peered":"2024-09-18T00:58:05.053230+0000","last_clean":"2024-09-18T00:58:05.053230+0000","last_became_active":"2024-09-18T00:57:27.311231+0000","last_became_peered":"2024-09-18T00:57:27.311231+0000","last_unstale":"2024-09-18T00:58:05.053230+0000","last_undegraded":"2024-09-18T00:58:05.053230+0000","last_fullsized":"2024-09-18T00:58:05.053230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:57.340012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020990940000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"45'184","reported_seq":1299,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.367452+0000","last_change":"2024-09-18T00:57:38.920098+0000","last_active":"2024-09-18T00:58:05.367452+0000","last_peered":"2024-09-18T00:58:05.367452+0000","last_clean":"2024-09-18T00:58:05.367452+0000","last_became_active":"2024-09-18T00:57:36.920599+0000","last_became_peered":"2024-09-18T00:57:36.920599+0000","last_unstale":"2024-09-18T00:58:05.367452+0000","last_undegraded":"2024-09-18T00:58:05.367452+0000","last_fullsized":"2024-09-18T00:58:05.367452+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:35:24.275062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276634,"stat_sum":{"num_bytes":10629681,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":1669,"num_read_kb":14215,"num_write":257,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.049958+0000","last_change":"2024-09-18T00:57:29.327841+0000","last_active":"2024-09-18T00:58:05.049958+0000","last_peered":"2024-09-18T00:58:05.049958+0000","last_clean":"2024-09-18T00:58:05.049958+0000","last_became_active":"2024-09-18T00:57:27.315417+0000","last_became_peered":"2024-09-18T00:57:27.315417+0000","last_unstale":"2024-09-18T00:58:05.049958+0000","last_undegraded":"2024-09-18T00:58:05.049958+0000","last_fullsized":"2024-09-18T00:58:05.049958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:54.335334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"45'110","reported_seq":600,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.758816+0000","last_change":"2024-09-18T00:57:38.920634+0000","last_active":"2024-09-18T00:58:05.758816+0000","last_peered":"2024-09-18T00:58:05.758816+0000","last_clean":"2024-09-18T00:58:05.758816+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T00:58:05.758816+0000","last_undegraded":"2024-09-18T00:58:05.758816+0000","last_fullsized":"2024-09-18T00:58:05.758816+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:52:11.987816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728301,"stat_sum":{"num_bytes":3756895,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":15,"num_read":850,"num_read_kb":1300,"num_write":85,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.052709+0000","last_change":"2024-09-18T00:57:29.327995+0000","last_active":"2024-09-18T00:58:05.052709+0000","last_peered":"2024-09-18T00:58:05.052709+0000","last_clean":"2024-09-18T00:58:05.052709+0000","last_became_active":"2024-09-18T00:57:27.316238+0000","last_became_peered":"2024-09-18T00:57:27.316238+0000","last_unstale":"2024-09-18T00:58:05.052709+0000","last_undegraded":"2024-09-18T00:58:05.052709+0000","last_fullsized":"2024-09-18T00:58:05.052709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019441149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"45'83","reported_seq":455,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.502450+0000","last_change":"2024-09-18T00:57:39.056519+0000","last_active":"2024-09-18T00:58:05.502450+0000","last_peered":"2024-09-18T00:58:05.502450+0000","last_clean":"2024-09-18T00:58:05.502450+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T00:58:05.502450+0000","last_undegraded":"2024-09-18T00:58:05.502450+0000","last_fullsized":"2024-09-18T00:58:05.502450+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:21:49.021432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043106399999999999,"stat_sum":{"num_bytes":10021233,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":15,"num_read":342,"num_read_kb":3267,"num_write":72,"num_write_kb":7863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.071560+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:58:05.071560+0000","last_peered":"2024-09-18T00:58:05.071560+0000","last_clean":"2024-09-18T00:58:05.071560+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:58:05.071560+0000","last_undegraded":"2024-09-18T00:58:05.071560+0000","last_fullsized":"2024-09-18T00:58:05.071560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"45'80","reported_seq":620,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.389665+0000","last_change":"2024-09-18T00:57:39.381374+0000","last_active":"2024-09-18T00:58:05.389665+0000","last_peered":"2024-09-18T00:58:05.389665+0000","last_clean":"2024-09-18T00:58:05.389665+0000","last_became_active":"2024-09-18T00:57:36.902781+0000","last_became_peered":"2024-09-18T00:57:36.902781+0000","last_unstale":"2024-09-18T00:58:05.389665+0000","last_undegraded":"2024-09-18T00:58:05.389665+0000","last_fullsized":"2024-09-18T00:58:05.389665+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:39:18.311233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000584944,"stat_sum":{"num_bytes":3586770,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":783,"num_read_kb":9661,"num_write":71,"num_write_kb":686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"45'208","reported_seq":1076,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.640205+0000","last_change":"2024-09-18T00:57:38.909273+0000","last_active":"2024-09-18T00:58:05.640205+0000","last_peered":"2024-09-18T00:58:05.640205+0000","last_clean":"2024-09-18T00:58:05.640205+0000","last_became_active":"2024-09-18T00:57:36.906264+0000","last_became_peered":"2024-09-18T00:57:36.906264+0000","last_unstale":"2024-09-18T00:58:05.640205+0000","last_undegraded":"2024-09-18T00:58:05.640205+0000","last_fullsized":"2024-09-18T00:58:05.640205+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:16.112366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056105599999999999,"stat_sum":{"num_bytes":25027841,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":1543,"num_read_kb":1691,"num_write":305,"num_write_kb":15813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":65,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.052120+0000","last_change":"2024-09-18T00:57:29.327436+0000","last_active":"2024-09-18T00:58:05.052120+0000","last_peered":"2024-09-18T00:58:05.052120+0000","last_clean":"2024-09-18T00:58:05.052120+0000","last_became_active":"2024-09-18T00:57:27.410600+0000","last_became_peered":"2024-09-18T00:57:27.410600+0000","last_unstale":"2024-09-18T00:58:05.052120+0000","last_undegraded":"2024-09-18T00:58:05.052120+0000","last_fullsized":"2024-09-18T00:58:05.052120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:59.380173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"45'179","reported_seq":485,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.555170+0000","last_change":"2024-09-18T00:57:38.907753+0000","last_active":"2024-09-18T00:58:05.555170+0000","last_peered":"2024-09-18T00:58:05.555170+0000","last_clean":"2024-09-18T00:58:05.555170+0000","last_became_active":"2024-09-18T00:57:36.907660+0000","last_became_peered":"2024-09-18T00:57:36.907660+0000","last_unstale":"2024-09-18T00:58:05.555170+0000","last_undegraded":"2024-09-18T00:58:05.555170+0000","last_fullsized":"2024-09-18T00:58:05.555170+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:58:51.966720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090282999999999997,"stat_sum":{"num_bytes":9302894,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":794,"num_read_kb":4574,"num_write":236,"num_write_kb":1020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":69,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T00:58:08.077331+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:58:08.077331+0000","last_peered":"2024-09-18T00:58:08.077331+0000","last_clean":"2024-09-18T00:58:08.077331+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:58:08.077331+0000","last_undegraded":"2024-09-18T00:58:08.077331+0000","last_fullsized":"2024-09-18T00:58:08.077331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"45'81","reported_seq":839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.717463+0000","last_change":"2024-09-18T00:57:39.056604+0000","last_active":"2024-09-18T00:58:05.717463+0000","last_peered":"2024-09-18T00:58:05.717463+0000","last_clean":"2024-09-18T00:58:05.717463+0000","last_became_active":"2024-09-18T00:57:36.903081+0000","last_became_peered":"2024-09-18T00:57:36.903081+0000","last_unstale":"2024-09-18T00:58:05.717463+0000","last_undegraded":"2024-09-18T00:58:05.717463+0000","last_fullsized":"2024-09-18T00:58:05.717463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:56.242765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045206800000000002,"stat_sum":{"num_bytes":9486939,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":728,"num_read_kb":11670,"num_write":67,"num_write_kb":883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.051540+0000","last_change":"2024-09-18T00:57:29.327276+0000","last_active":"2024-09-18T00:58:05.051540+0000","last_peered":"2024-09-18T00:58:05.051540+0000","last_clean":"2024-09-18T00:58:05.051540+0000","last_became_active":"2024-09-18T00:57:27.316217+0000","last_became_peered":"2024-09-18T00:57:27.316217+0000","last_unstale":"2024-09-18T00:58:05.051540+0000","last_undegraded":"2024-09-18T00:58:05.051540+0000","last_fullsized":"2024-09-18T00:58:05.051540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:38.026720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012774469999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"45'95","reported_seq":324,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.194743+0000","last_change":"2024-09-18T00:57:38.919998+0000","last_active":"2024-09-18T00:58:05.194743+0000","last_peered":"2024-09-18T00:58:05.194743+0000","last_clean":"2024-09-18T00:58:05.194743+0000","last_became_active":"2024-09-18T00:57:36.906680+0000","last_became_peered":"2024-09-18T00:57:36.906680+0000","last_unstale":"2024-09-18T00:58:05.194743+0000","last_undegraded":"2024-09-18T00:58:05.194743+0000","last_fullsized":"2024-09-18T00:58:05.194743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:18:17.220539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064609,"stat_sum":{"num_bytes":9585705,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":15,"num_read":196,"num_read_kb":2168,"num_write":85,"num_write_kb":33631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.052271+0000","last_change":"2024-09-18T00:57:29.328733+0000","last_active":"2024-09-18T00:58:05.052271+0000","last_peered":"2024-09-18T00:58:05.052271+0000","last_clean":"2024-09-18T00:58:05.052271+0000","last_became_active":"2024-09-18T00:57:27.410396+0000","last_became_peered":"2024-09-18T00:57:27.410396+0000","last_unstale":"2024-09-18T00:58:05.052271+0000","last_undegraded":"2024-09-18T00:58:05.052271+0000","last_fullsized":"2024-09-18T00:58:05.052271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:27.438117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":214,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.057073+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:58:05.057073+0000","last_peered":"2024-09-18T00:58:05.057073+0000","last_clean":"2024-09-18T00:58:05.057073+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:58:05.057073+0000","last_undegraded":"2024-09-18T00:58:05.057073+0000","last_fullsized":"2024-09-18T00:58:05.057073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"45'78","reported_seq":729,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.644341+0000","last_change":"2024-09-18T00:57:38.919534+0000","last_active":"2024-09-18T00:58:05.644341+0000","last_peered":"2024-09-18T00:58:05.644341+0000","last_clean":"2024-09-18T00:58:05.644341+0000","last_became_active":"2024-09-18T00:57:36.908955+0000","last_became_peered":"2024-09-18T00:57:36.908955+0000","last_unstale":"2024-09-18T00:58:05.644341+0000","last_undegraded":"2024-09-18T00:58:05.644341+0000","last_fullsized":"2024-09-18T00:58:05.644341+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:48:39.909894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077760500000000001,"stat_sum":{"num_bytes":15125012,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":695,"num_read_kb":14139,"num_write":89,"num_write_kb":11286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"44'82","reported_seq":583,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.051929+0000","last_change":"2024-09-18T00:57:38.899513+0000","last_active":"2024-09-18T00:58:05.051929+0000","last_peered":"2024-09-18T00:58:05.051929+0000","last_clean":"2024-09-18T00:58:05.051929+0000","last_became_active":"2024-09-18T00:57:36.911337+0000","last_became_peered":"2024-09-18T00:57:36.911337+0000","last_unstale":"2024-09-18T00:58:05.051929+0000","last_undegraded":"2024-09-18T00:58:05.051929+0000","last_fullsized":"2024-09-18T00:58:05.051929+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:32:34.038951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061570699999999995,"stat_sum":{"num_bytes":9928288,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":12,"num_read":464,"num_read_kb":2983,"num_write":68,"num_write_kb":7916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"44'91","reported_seq":762,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.052044+0000","last_change":"2024-09-18T00:57:38.899904+0000","last_active":"2024-09-18T00:58:05.052044+0000","last_peered":"2024-09-18T00:58:05.052044+0000","last_clean":"2024-09-18T00:58:05.052044+0000","last_became_active":"2024-09-18T00:57:36.901837+0000","last_became_peered":"2024-09-18T00:57:36.901837+0000","last_unstale":"2024-09-18T00:58:05.052044+0000","last_undegraded":"2024-09-18T00:58:05.052044+0000","last_fullsized":"2024-09-18T00:58:05.052044+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:30.415538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097047300000000002,"stat_sum":{"num_bytes":6788077,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":657,"num_read_kb":11949,"num_write":68,"num_write_kb":684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"48'99","reported_seq":1542,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T00:58:08.646238+0000","last_change":"2024-09-18T00:57:38.899763+0000","last_active":"2024-09-18T00:58:08.646238+0000","last_peered":"2024-09-18T00:58:08.646238+0000","last_clean":"2024-09-18T00:58:08.646238+0000","last_became_active":"2024-09-18T00:57:36.905869+0000","last_became_peered":"2024-09-18T00:57:36.905869+0000","last_unstale":"2024-09-18T00:58:08.646238+0000","last_undegraded":"2024-09-18T00:58:08.646238+0000","last_fullsized":"2024-09-18T00:58:08.646238+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:13.804442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097615600000000005,"stat_sum":{"num_bytes":11850172,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":1601,"num_read_kb":21540,"num_write":70,"num_write_kb":1353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"45'103","reported_seq":537,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.429632+0000","last_change":"2024-09-18T00:57:38.909165+0000","last_active":"2024-09-18T00:58:05.429632+0000","last_peered":"2024-09-18T00:58:05.429632+0000","last_clean":"2024-09-18T00:58:05.429632+0000","last_became_active":"2024-09-18T00:57:36.908288+0000","last_became_peered":"2024-09-18T00:57:36.908288+0000","last_unstale":"2024-09-18T00:58:05.429632+0000","last_undegraded":"2024-09-18T00:58:05.429632+0000","last_fullsized":"2024-09-18T00:58:05.429632+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:42:10.554772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044375399999999998,"stat_sum":{"num_bytes":5595918,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":590,"num_read_kb":7033,"num_write":132,"num_write_kb":1219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"45'371","reported_seq":1322,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.806677+0000","last_change":"2024-09-18T00:57:38.903046+0000","last_active":"2024-09-18T00:58:05.806677+0000","last_peered":"2024-09-18T00:58:05.806677+0000","last_clean":"2024-09-18T00:58:05.806677+0000","last_became_active":"2024-09-18T00:57:36.900286+0000","last_became_peered":"2024-09-18T00:57:36.900286+0000","last_unstale":"2024-09-18T00:58:05.806677+0000","last_undegraded":"2024-09-18T00:58:05.806677+0000","last_fullsized":"2024-09-18T00:58:05.806677+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:14.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023201229999999999,"stat_sum":{"num_bytes":4929779,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":16,"num_read":3102,"num_read_kb":8018,"num_write":511,"num_write_kb":972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"45'94","reported_seq":380,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.452921+0000","last_change":"2024-09-18T00:57:39.382152+0000","last_active":"2024-09-18T00:58:05.452921+0000","last_peered":"2024-09-18T00:58:05.452921+0000","last_clean":"2024-09-18T00:58:05.452921+0000","last_became_active":"2024-09-18T00:57:36.904791+0000","last_became_peered":"2024-09-18T00:57:36.904791+0000","last_unstale":"2024-09-18T00:58:05.452921+0000","last_undegraded":"2024-09-18T00:58:05.452921+0000","last_fullsized":"2024-09-18T00:58:05.452921+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:15:10.983235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090629000000000003,"stat_sum":{"num_bytes":942702,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":606,"num_read_kb":738,"num_write":81,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"45'166","reported_seq":493,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.716346+0000","last_change":"2024-09-18T00:57:38.909597+0000","last_active":"2024-09-18T00:58:05.716346+0000","last_peered":"2024-09-18T00:58:05.716346+0000","last_clean":"2024-09-18T00:58:05.716346+0000","last_became_active":"2024-09-18T00:57:36.905462+0000","last_became_peered":"2024-09-18T00:57:36.905462+0000","last_unstale":"2024-09-18T00:58:05.716346+0000","last_undegraded":"2024-09-18T00:58:05.716346+0000","last_fullsized":"2024-09-18T00:58:05.716346+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:24:59.482353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081831799999999995,"stat_sum":{"num_bytes":9001370,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":799,"num_read_kb":6628,"num_write":218,"num_write_kb":4285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"45'144","reported_seq":1799,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.221446+0000","last_change":"2024-09-18T00:57:38.900906+0000","last_active":"2024-09-18T00:58:05.221446+0000","last_peered":"2024-09-18T00:58:05.221446+0000","last_clean":"2024-09-18T00:58:05.221446+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T00:58:05.221446+0000","last_undegraded":"2024-09-18T00:58:05.221446+0000","last_fullsized":"2024-09-18T00:58:05.221446+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:21:18.073028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069386899999999995,"stat_sum":{"num_bytes":30277407,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":2226,"num_read_kb":33163,"num_write":125,"num_write_kb":23122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"45'81","reported_seq":216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.643573+0000","last_change":"2024-09-18T00:57:38.899297+0000","last_active":"2024-09-18T00:58:05.643573+0000","last_peered":"2024-09-18T00:58:05.643573+0000","last_clean":"2024-09-18T00:58:05.643573+0000","last_became_active":"2024-09-18T00:57:36.897289+0000","last_became_peered":"2024-09-18T00:57:36.897289+0000","last_unstale":"2024-09-18T00:58:05.643573+0000","last_undegraded":"2024-09-18T00:58:05.643573+0000","last_fullsized":"2024-09-18T00:58:05.643573+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:04:03.352923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071487799999999995,"stat_sum":{"num_bytes":12846580,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":91,"num_read_kb":920,"num_write":64,"num_write_kb":823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"45'110","reported_seq":929,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.782935+0000","last_change":"2024-09-18T00:57:39.056761+0000","last_active":"2024-09-18T00:58:05.782935+0000","last_peered":"2024-09-18T00:58:05.782935+0000","last_clean":"2024-09-18T00:58:05.782935+0000","last_became_active":"2024-09-18T00:57:36.910991+0000","last_became_peered":"2024-09-18T00:57:36.910991+0000","last_unstale":"2024-09-18T00:58:05.782935+0000","last_undegraded":"2024-09-18T00:58:05.782935+0000","last_fullsized":"2024-09-18T00:58:05.782935+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:02:07.471326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055104700000000004,"stat_sum":{"num_bytes":11309164,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":986,"num_read_kb":9006,"num_write":130,"num_write_kb":8123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"45'194","reported_seq":666,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.280048+0000","last_change":"2024-09-18T00:57:38.899783+0000","last_active":"2024-09-18T00:58:05.280048+0000","last_peered":"2024-09-18T00:58:05.280048+0000","last_clean":"2024-09-18T00:58:05.280048+0000","last_became_active":"2024-09-18T00:57:36.904999+0000","last_became_peered":"2024-09-18T00:57:36.904999+0000","last_unstale":"2024-09-18T00:58:05.280048+0000","last_undegraded":"2024-09-18T00:58:05.280048+0000","last_fullsized":"2024-09-18T00:58:05.280048+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:29:41.433800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070399799999999995,"stat_sum":{"num_bytes":1572910,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":1322,"num_read_kb":3419,"num_write":254,"num_write_kb":1370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":342236749,"num_objects":855,"num_object_clones":134,"num_object_copies":1710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":855,"num_whiteouts":430,"num_read":37185,"num_read_kb":254658,"num_write":5301,"num_write_kb":236164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":234348544,"data_stored":247296514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4490,"ondisk_log_size":4490,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61284,"kb_used_data":25932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94310556,"statfs":{"total":96636764160,"available":96574009344,"internally_reserved":0,"allocated":26554368,"data_stored":26414261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71744,"kb_used_data":36388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94300096,"statfs":{"total":96636764160,"available":96563298304,"internally_reserved":0,"allocated":37261312,"data_stored":38449144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71596,"kb_used_data":36252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94300244,"statfs":{"total":96636764160,"available":96563449856,"internally_reserved":0,"allocated":37122048,"data_stored":38472462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64104,"kb_used_data":24600,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94307736,"statfs":{"total":96636764160,"available":96571121664,"internally_reserved":0,"allocated":25190400,"data_stored":28109470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56780,"kb_used_data":21436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94315060,"statfs":{"total":96636764160,"available":96578621440,"internally_reserved":0,"allocated":21950464,"data_stored":23380080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72484,"kb_used_data":37140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94299356,"statfs":{"total":96636764160,"available":96562540544,"internally_reserved":0,"allocated":38031360,"data_stored":39154376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77348,"kb_used_data":37840,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94294492,"statfs":{"total":96636764160,"available":96557559808,"internally_reserved":0,"allocated":38748160,"data_stored":40140790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49436,"kb_used_data":14092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322404,"statfs":{"total":96636764160,"available":96586141696,"internally_reserved":0,"allocated":14430208,"data_stored":15794271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":38277120,"data_stored":39949840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36995072,"data_stored":38408539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21467136,"data_stored":23183349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24682496,"data_stored":27900343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36773888,"data_stored":38393113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36134912,"data_stored":37623910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":26116096,"data_stored":26267499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13901824,"data_stored":15569921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:58:10.479 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:58:10.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:10.522 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:10.518+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:10.595 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:10.590+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:10.645 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:10.645 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:10.659 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:10.661 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:10.703 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:10.698+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:10.796 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:10.794+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:10.819 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:10.819 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:10.837 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:10.840 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:58:10.840 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:58:10.849 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:10.849 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:10.861 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-18T00:58:10.097340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":342827136,"num_objects":859,"num_object_clones":134,"num_object_copies":1718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":859,"num_whiteouts":430,"num_read":37301,"num_read_kb":254758,"num_write":5462,"num_write_kb":238582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4610,"ondisk_log_size":4610,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":524776,"kb_used_data":233680,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754449944,"statfs":{"total":773094113280,"available":772556742656,"internally_reserved":0,"allocated":239288320,"data_stored":249914854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131072,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":63,"num_read_kb":304,"num_write":8,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.037653"},"pg_stats":[{"pgid":"3.1f","version":"45'108","reported_seq":663,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.830008+0000","last_change":"2024-09-18T00:57:38.909849+0000","last_active":"2024-09-18T00:58:05.830008+0000","last_peered":"2024-09-18T00:58:05.830008+0000","last_clean":"2024-09-18T00:58:05.830008+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T00:58:05.830008+0000","last_undegraded":"2024-09-18T00:58:05.830008+0000","last_fullsized":"2024-09-18T00:58:05.830008+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:40:00.440095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001565786,"stat_sum":{"num_bytes":1474576,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":14,"num_read":982,"num_read_kb":2293,"num_write":92,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"45'83","reported_seq":826,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.694076+0000","last_change":"2024-09-18T00:57:38.908434+0000","last_active":"2024-09-18T00:58:05.694076+0000","last_peered":"2024-09-18T00:58:05.694076+0000","last_clean":"2024-09-18T00:58:05.694076+0000","last_became_active":"2024-09-18T00:57:36.897948+0000","last_became_peered":"2024-09-18T00:57:36.897948+0000","last_unstale":"2024-09-18T00:58:05.694076+0000","last_undegraded":"2024-09-18T00:58:05.694076+0000","last_fullsized":"2024-09-18T00:58:05.694076+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:42:05.883003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040574399999999998,"stat_sum":{"num_bytes":5035983,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":19,"num_read":720,"num_read_kb":4752,"num_write":53,"num_write_kb":303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"45'293","reported_seq":974,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.133005+0000","last_change":"2024-09-18T00:57:38.902294+0000","last_active":"2024-09-18T00:58:05.133005+0000","last_peered":"2024-09-18T00:58:05.133005+0000","last_clean":"2024-09-18T00:58:05.133005+0000","last_became_active":"2024-09-18T00:57:36.906790+0000","last_became_peered":"2024-09-18T00:57:36.906790+0000","last_unstale":"2024-09-18T00:58:05.133005+0000","last_undegraded":"2024-09-18T00:58:05.133005+0000","last_fullsized":"2024-09-18T00:58:05.133005+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:34:19.660233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039089,"stat_sum":{"num_bytes":8240308,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":1757,"num_read_kb":12004,"num_write":453,"num_write_kb":1708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"45'282","reported_seq":1226,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.240177+0000","last_change":"2024-09-18T00:57:38.901600+0000","last_active":"2024-09-18T00:58:05.240177+0000","last_peered":"2024-09-18T00:58:05.240177+0000","last_clean":"2024-09-18T00:58:05.240177+0000","last_became_active":"2024-09-18T00:57:36.906003+0000","last_became_peered":"2024-09-18T00:57:36.906003+0000","last_unstale":"2024-09-18T00:58:05.240177+0000","last_undegraded":"2024-09-18T00:58:05.240177+0000","last_fullsized":"2024-09-18T00:58:05.240177+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:47.263915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000711266,"stat_sum":{"num_bytes":2320457,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":2243,"num_read_kb":8362,"num_write":349,"num_write_kb":1522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"45'287","reported_seq":1733,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.641256+0000","last_change":"2024-09-18T00:57:38.921854+0000","last_active":"2024-09-18T00:58:05.641256+0000","last_peered":"2024-09-18T00:58:05.641256+0000","last_clean":"2024-09-18T00:58:05.641256+0000","last_became_active":"2024-09-18T00:57:36.913393+0000","last_became_peered":"2024-09-18T00:57:36.913393+0000","last_unstale":"2024-09-18T00:58:05.641256+0000","last_undegraded":"2024-09-18T00:58:05.641256+0000","last_fullsized":"2024-09-18T00:58:05.641256+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:49:14.675049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002554022,"stat_sum":{"num_bytes":14881739,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":2406,"num_read_kb":9573,"num_write":423,"num_write_kb":8944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"45'132","reported_seq":1286,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.799313+0000","last_change":"2024-09-18T00:57:38.901805+0000","last_active":"2024-09-18T00:58:05.799313+0000","last_peered":"2024-09-18T00:58:05.799313+0000","last_clean":"2024-09-18T00:58:05.799313+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T00:58:05.799313+0000","last_undegraded":"2024-09-18T00:58:05.799313+0000","last_fullsized":"2024-09-18T00:58:05.799313+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:09.880267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098070899999999996,"stat_sum":{"num_bytes":26226007,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":1854,"num_read_kb":7668,"num_write":108,"num_write_kb":605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"45'86","reported_seq":1797,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.400596+0000","last_change":"2024-09-18T00:57:38.922010+0000","last_active":"2024-09-18T00:58:05.400596+0000","last_peered":"2024-09-18T00:58:05.400596+0000","last_clean":"2024-09-18T00:58:05.400596+0000","last_became_active":"2024-09-18T00:57:36.910457+0000","last_became_peered":"2024-09-18T00:57:36.910457+0000","last_unstale":"2024-09-18T00:58:05.400596+0000","last_undegraded":"2024-09-18T00:58:05.400596+0000","last_fullsized":"2024-09-18T00:58:05.400596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:55.403260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026842849999999998,"stat_sum":{"num_bytes":14881824,"num_objects":29,"num_object_clones":3,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":1693,"num_read_kb":9073,"num_write":94,"num_write_kb":8711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"45'163","reported_seq":1016,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.644876+0000","last_change":"2024-09-18T00:57:38.908144+0000","last_active":"2024-09-18T00:58:05.644876+0000","last_peered":"2024-09-18T00:58:05.644876+0000","last_clean":"2024-09-18T00:58:05.644876+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T00:58:05.644876+0000","last_undegraded":"2024-09-18T00:58:05.644876+0000","last_fullsized":"2024-09-18T00:58:05.644876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:04.349457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000871597,"stat_sum":{"num_bytes":28436710,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":1428,"num_read_kb":1474,"num_write":175,"num_write_kb":33827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"45'106","reported_seq":649,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.606953+0000","last_change":"2024-09-18T00:57:38.910038+0000","last_active":"2024-09-18T00:58:05.606953+0000","last_peered":"2024-09-18T00:58:05.606953+0000","last_clean":"2024-09-18T00:58:05.606953+0000","last_became_active":"2024-09-18T00:57:36.895915+0000","last_became_peered":"2024-09-18T00:57:36.895915+0000","last_unstale":"2024-09-18T00:58:05.606953+0000","last_undegraded":"2024-09-18T00:58:05.606953+0000","last_fullsized":"2024-09-18T00:58:05.606953+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:23:48.528113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001269769,"stat_sum":{"num_bytes":7187217,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":805,"num_read_kb":11695,"num_write":93,"num_write_kb":4072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"45'166","reported_seq":2411,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.132073+0000","last_change":"2024-09-18T00:57:39.381845+0000","last_active":"2024-09-18T00:58:05.132073+0000","last_peered":"2024-09-18T00:58:05.132073+0000","last_clean":"2024-09-18T00:58:05.132073+0000","last_became_active":"2024-09-18T00:57:36.894538+0000","last_became_peered":"2024-09-18T00:57:36.894538+0000","last_unstale":"2024-09-18T00:58:05.132073+0000","last_undegraded":"2024-09-18T00:58:05.132073+0000","last_fullsized":"2024-09-18T00:58:05.132073+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:28:34.626070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076552,"stat_sum":{"num_bytes":7716198,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":2765,"num_read_kb":12240,"num_write":300,"num_write_kb":14789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"45'111","reported_seq":591,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.642020+0000","last_change":"2024-09-18T00:57:38.909647+0000","last_active":"2024-09-18T00:58:05.642020+0000","last_peered":"2024-09-18T00:58:05.642020+0000","last_clean":"2024-09-18T00:58:05.642020+0000","last_became_active":"2024-09-18T00:57:36.902977+0000","last_became_peered":"2024-09-18T00:57:36.902977+0000","last_unstale":"2024-09-18T00:58:05.642020+0000","last_undegraded":"2024-09-18T00:58:05.642020+0000","last_fullsized":"2024-09-18T00:58:05.642020+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:40:26.460414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051351900000000004,"stat_sum":{"num_bytes":23482430,"num_objects":29,"num_object_clones":3,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":17,"num_read":446,"num_read_kb":7407,"num_write":140,"num_write_kb":38140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"45'40","reported_seq":130,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.584297+0000","last_change":"2024-09-18T00:57:38.902578+0000","last_active":"2024-09-18T00:58:05.584297+0000","last_peered":"2024-09-18T00:58:05.584297+0000","last_clean":"2024-09-18T00:58:05.584297+0000","last_became_active":"2024-09-18T00:57:36.894977+0000","last_became_peered":"2024-09-18T00:57:36.894977+0000","last_unstale":"2024-09-18T00:58:05.584297+0000","last_undegraded":"2024-09-18T00:58:05.584297+0000","last_fullsized":"2024-09-18T00:58:05.584297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:33:26.607066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028223319999999999,"stat_sum":{"num_bytes":787963,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":6,"num_read":42,"num_read_kb":35,"num_write":33,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.053230+0000","last_change":"2024-09-18T00:57:29.328163+0000","last_active":"2024-09-18T00:58:05.053230+0000","last_peered":"2024-09-18T00:58:05.053230+0000","last_clean":"2024-09-18T00:58:05.053230+0000","last_became_active":"2024-09-18T00:57:27.311231+0000","last_became_peered":"2024-09-18T00:57:27.311231+0000","last_unstale":"2024-09-18T00:58:05.053230+0000","last_undegraded":"2024-09-18T00:58:05.053230+0000","last_fullsized":"2024-09-18T00:58:05.053230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:57.340012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020990940000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"45'184","reported_seq":1299,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.367452+0000","last_change":"2024-09-18T00:57:38.920098+0000","last_active":"2024-09-18T00:58:05.367452+0000","last_peered":"2024-09-18T00:58:05.367452+0000","last_clean":"2024-09-18T00:58:05.367452+0000","last_became_active":"2024-09-18T00:57:36.920599+0000","last_became_peered":"2024-09-18T00:57:36.920599+0000","last_unstale":"2024-09-18T00:58:05.367452+0000","last_undegraded":"2024-09-18T00:58:05.367452+0000","last_fullsized":"2024-09-18T00:58:05.367452+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:35:24.275062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276634,"stat_sum":{"num_bytes":10629681,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":1669,"num_read_kb":14215,"num_write":257,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.049958+0000","last_change":"2024-09-18T00:57:29.327841+0000","last_active":"2024-09-18T00:58:05.049958+0000","last_peered":"2024-09-18T00:58:05.049958+0000","last_clean":"2024-09-18T00:58:05.049958+0000","last_became_active":"2024-09-18T00:57:27.315417+0000","last_became_peered":"2024-09-18T00:57:27.315417+0000","last_unstale":"2024-09-18T00:58:05.049958+0000","last_undegraded":"2024-09-18T00:58:05.049958+0000","last_fullsized":"2024-09-18T00:58:05.049958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:54.335334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"45'110","reported_seq":600,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.758816+0000","last_change":"2024-09-18T00:57:38.920634+0000","last_active":"2024-09-18T00:58:05.758816+0000","last_peered":"2024-09-18T00:58:05.758816+0000","last_clean":"2024-09-18T00:58:05.758816+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T00:58:05.758816+0000","last_undegraded":"2024-09-18T00:58:05.758816+0000","last_fullsized":"2024-09-18T00:58:05.758816+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:52:11.987816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728301,"stat_sum":{"num_bytes":3756895,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":15,"num_read":850,"num_read_kb":1300,"num_write":85,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.052709+0000","last_change":"2024-09-18T00:57:29.327995+0000","last_active":"2024-09-18T00:58:05.052709+0000","last_peered":"2024-09-18T00:58:05.052709+0000","last_clean":"2024-09-18T00:58:05.052709+0000","last_became_active":"2024-09-18T00:57:27.316238+0000","last_became_peered":"2024-09-18T00:57:27.316238+0000","last_unstale":"2024-09-18T00:58:05.052709+0000","last_undegraded":"2024-09-18T00:58:05.052709+0000","last_fullsized":"2024-09-18T00:58:05.052709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019441149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"45'83","reported_seq":455,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.502450+0000","last_change":"2024-09-18T00:57:39.056519+0000","last_active":"2024-09-18T00:58:05.502450+0000","last_peered":"2024-09-18T00:58:05.502450+0000","last_clean":"2024-09-18T00:58:05.502450+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T00:58:05.502450+0000","last_undegraded":"2024-09-18T00:58:05.502450+0000","last_fullsized":"2024-09-18T00:58:05.502450+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:21:49.021432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043106399999999999,"stat_sum":{"num_bytes":10021233,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":15,"num_read":342,"num_read_kb":3267,"num_write":72,"num_write_kb":7863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.071560+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:58:05.071560+0000","last_peered":"2024-09-18T00:58:05.071560+0000","last_clean":"2024-09-18T00:58:05.071560+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:58:05.071560+0000","last_undegraded":"2024-09-18T00:58:05.071560+0000","last_fullsized":"2024-09-18T00:58:05.071560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"45'80","reported_seq":620,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.389665+0000","last_change":"2024-09-18T00:57:39.381374+0000","last_active":"2024-09-18T00:58:05.389665+0000","last_peered":"2024-09-18T00:58:05.389665+0000","last_clean":"2024-09-18T00:58:05.389665+0000","last_became_active":"2024-09-18T00:57:36.902781+0000","last_became_peered":"2024-09-18T00:57:36.902781+0000","last_unstale":"2024-09-18T00:58:05.389665+0000","last_undegraded":"2024-09-18T00:58:05.389665+0000","last_fullsized":"2024-09-18T00:58:05.389665+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:39:18.311233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000584944,"stat_sum":{"num_bytes":3586770,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":783,"num_read_kb":9661,"num_write":71,"num_write_kb":686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"45'208","reported_seq":1076,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.640205+0000","last_change":"2024-09-18T00:57:38.909273+0000","last_active":"2024-09-18T00:58:05.640205+0000","last_peered":"2024-09-18T00:58:05.640205+0000","last_clean":"2024-09-18T00:58:05.640205+0000","last_became_active":"2024-09-18T00:57:36.906264+0000","last_became_peered":"2024-09-18T00:57:36.906264+0000","last_unstale":"2024-09-18T00:58:05.640205+0000","last_undegraded":"2024-09-18T00:58:05.640205+0000","last_fullsized":"2024-09-18T00:58:05.640205+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:16.112366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056105599999999999,"stat_sum":{"num_bytes":25027841,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":1543,"num_read_kb":1691,"num_write":305,"num_write_kb":15813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":65,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.052120+0000","last_change":"2024-09-18T00:57:29.327436+0000","last_active":"2024-09-18T00:58:05.052120+0000","last_peered":"2024-09-18T00:58:05.052120+0000","last_clean":"2024-09-18T00:58:05.052120+0000","last_became_active":"2024-09-18T00:57:27.410600+0000","last_became_peered":"2024-09-18T00:57:27.410600+0000","last_unstale":"2024-09-18T00:58:05.052120+0000","last_undegraded":"2024-09-18T00:58:05.052120+0000","last_fullsized":"2024-09-18T00:58:05.052120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:59.380173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"45'179","reported_seq":485,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.555170+0000","last_change":"2024-09-18T00:57:38.907753+0000","last_active":"2024-09-18T00:58:05.555170+0000","last_peered":"2024-09-18T00:58:05.555170+0000","last_clean":"2024-09-18T00:58:05.555170+0000","last_became_active":"2024-09-18T00:57:36.907660+0000","last_became_peered":"2024-09-18T00:57:36.907660+0000","last_unstale":"2024-09-18T00:58:05.555170+0000","last_undegraded":"2024-09-18T00:58:05.555170+0000","last_fullsized":"2024-09-18T00:58:05.555170+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:58:51.966720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090282999999999997,"stat_sum":{"num_bytes":9302894,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":794,"num_read_kb":4574,"num_write":236,"num_write_kb":1020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":69,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T00:58:08.077331+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:58:08.077331+0000","last_peered":"2024-09-18T00:58:08.077331+0000","last_clean":"2024-09-18T00:58:08.077331+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:58:08.077331+0000","last_undegraded":"2024-09-18T00:58:08.077331+0000","last_fullsized":"2024-09-18T00:58:08.077331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"45'81","reported_seq":839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.717463+0000","last_change":"2024-09-18T00:57:39.056604+0000","last_active":"2024-09-18T00:58:05.717463+0000","last_peered":"2024-09-18T00:58:05.717463+0000","last_clean":"2024-09-18T00:58:05.717463+0000","last_became_active":"2024-09-18T00:57:36.903081+0000","last_became_peered":"2024-09-18T00:57:36.903081+0000","last_unstale":"2024-09-18T00:58:05.717463+0000","last_undegraded":"2024-09-18T00:58:05.717463+0000","last_fullsized":"2024-09-18T00:58:05.717463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:56.242765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045206800000000002,"stat_sum":{"num_bytes":9486939,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":728,"num_read_kb":11670,"num_write":67,"num_write_kb":883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.051540+0000","last_change":"2024-09-18T00:57:29.327276+0000","last_active":"2024-09-18T00:58:05.051540+0000","last_peered":"2024-09-18T00:58:05.051540+0000","last_clean":"2024-09-18T00:58:05.051540+0000","last_became_active":"2024-09-18T00:57:27.316217+0000","last_became_peered":"2024-09-18T00:57:27.316217+0000","last_unstale":"2024-09-18T00:58:05.051540+0000","last_undegraded":"2024-09-18T00:58:05.051540+0000","last_fullsized":"2024-09-18T00:58:05.051540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:38.026720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012774469999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"45'95","reported_seq":324,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.194743+0000","last_change":"2024-09-18T00:57:38.919998+0000","last_active":"2024-09-18T00:58:05.194743+0000","last_peered":"2024-09-18T00:58:05.194743+0000","last_clean":"2024-09-18T00:58:05.194743+0000","last_became_active":"2024-09-18T00:57:36.906680+0000","last_became_peered":"2024-09-18T00:57:36.906680+0000","last_unstale":"2024-09-18T00:58:05.194743+0000","last_undegraded":"2024-09-18T00:58:05.194743+0000","last_fullsized":"2024-09-18T00:58:05.194743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:18:17.220539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064609,"stat_sum":{"num_bytes":9585705,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":15,"num_read":196,"num_read_kb":2168,"num_write":85,"num_write_kb":33631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.052271+0000","last_change":"2024-09-18T00:57:29.328733+0000","last_active":"2024-09-18T00:58:05.052271+0000","last_peered":"2024-09-18T00:58:05.052271+0000","last_clean":"2024-09-18T00:58:05.052271+0000","last_became_active":"2024-09-18T00:57:27.410396+0000","last_became_peered":"2024-09-18T00:57:27.410396+0000","last_unstale":"2024-09-18T00:58:05.052271+0000","last_undegraded":"2024-09-18T00:58:05.052271+0000","last_fullsized":"2024-09-18T00:58:05.052271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:27.438117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":214,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.057073+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:58:05.057073+0000","last_peered":"2024-09-18T00:58:05.057073+0000","last_clean":"2024-09-18T00:58:05.057073+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:58:05.057073+0000","last_undegraded":"2024-09-18T00:58:05.057073+0000","last_fullsized":"2024-09-18T00:58:05.057073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"45'78","reported_seq":729,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.644341+0000","last_change":"2024-09-18T00:57:38.919534+0000","last_active":"2024-09-18T00:58:05.644341+0000","last_peered":"2024-09-18T00:58:05.644341+0000","last_clean":"2024-09-18T00:58:05.644341+0000","last_became_active":"2024-09-18T00:57:36.908955+0000","last_became_peered":"2024-09-18T00:57:36.908955+0000","last_unstale":"2024-09-18T00:58:05.644341+0000","last_undegraded":"2024-09-18T00:58:05.644341+0000","last_fullsized":"2024-09-18T00:58:05.644341+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:48:39.909894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077760500000000001,"stat_sum":{"num_bytes":15125012,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":695,"num_read_kb":14139,"num_write":89,"num_write_kb":11286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"44'82","reported_seq":583,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.051929+0000","last_change":"2024-09-18T00:57:38.899513+0000","last_active":"2024-09-18T00:58:05.051929+0000","last_peered":"2024-09-18T00:58:05.051929+0000","last_clean":"2024-09-18T00:58:05.051929+0000","last_became_active":"2024-09-18T00:57:36.911337+0000","last_became_peered":"2024-09-18T00:57:36.911337+0000","last_unstale":"2024-09-18T00:58:05.051929+0000","last_undegraded":"2024-09-18T00:58:05.051929+0000","last_fullsized":"2024-09-18T00:58:05.051929+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:32:34.038951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061570699999999995,"stat_sum":{"num_bytes":9928288,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":12,"num_read":464,"num_read_kb":2983,"num_write":68,"num_write_kb":7916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"44'91","reported_seq":762,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.052044+0000","last_change":"2024-09-18T00:57:38.899904+0000","last_active":"2024-09-18T00:58:05.052044+0000","last_peered":"2024-09-18T00:58:05.052044+0000","last_clean":"2024-09-18T00:58:05.052044+0000","last_became_active":"2024-09-18T00:57:36.901837+0000","last_became_peered":"2024-09-18T00:57:36.901837+0000","last_unstale":"2024-09-18T00:58:05.052044+0000","last_undegraded":"2024-09-18T00:58:05.052044+0000","last_fullsized":"2024-09-18T00:58:05.052044+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:47:30.415538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097047300000000002,"stat_sum":{"num_bytes":6788077,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":657,"num_read_kb":11949,"num_write":68,"num_write_kb":684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"48'99","reported_seq":1542,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T00:58:08.646238+0000","last_change":"2024-09-18T00:57:38.899763+0000","last_active":"2024-09-18T00:58:08.646238+0000","last_peered":"2024-09-18T00:58:08.646238+0000","last_clean":"2024-09-18T00:58:08.646238+0000","last_became_active":"2024-09-18T00:57:36.905869+0000","last_became_peered":"2024-09-18T00:57:36.905869+0000","last_unstale":"2024-09-18T00:58:08.646238+0000","last_undegraded":"2024-09-18T00:58:08.646238+0000","last_fullsized":"2024-09-18T00:58:08.646238+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:13.804442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097615600000000005,"stat_sum":{"num_bytes":11850172,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":1601,"num_read_kb":21540,"num_write":70,"num_write_kb":1353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"45'103","reported_seq":537,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.429632+0000","last_change":"2024-09-18T00:57:38.909165+0000","last_active":"2024-09-18T00:58:05.429632+0000","last_peered":"2024-09-18T00:58:05.429632+0000","last_clean":"2024-09-18T00:58:05.429632+0000","last_became_active":"2024-09-18T00:57:36.908288+0000","last_became_peered":"2024-09-18T00:57:36.908288+0000","last_unstale":"2024-09-18T00:58:05.429632+0000","last_undegraded":"2024-09-18T00:58:05.429632+0000","last_fullsized":"2024-09-18T00:58:05.429632+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:42:10.554772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044375399999999998,"stat_sum":{"num_bytes":5595918,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":590,"num_read_kb":7033,"num_write":132,"num_write_kb":1219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"45'371","reported_seq":1322,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.806677+0000","last_change":"2024-09-18T00:57:38.903046+0000","last_active":"2024-09-18T00:58:05.806677+0000","last_peered":"2024-09-18T00:58:05.806677+0000","last_clean":"2024-09-18T00:58:05.806677+0000","last_became_active":"2024-09-18T00:57:36.900286+0000","last_became_peered":"2024-09-18T00:57:36.900286+0000","last_unstale":"2024-09-18T00:58:05.806677+0000","last_undegraded":"2024-09-18T00:58:05.806677+0000","last_fullsized":"2024-09-18T00:58:05.806677+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:14.218831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023201229999999999,"stat_sum":{"num_bytes":4929779,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":16,"num_read":3102,"num_read_kb":8018,"num_write":511,"num_write_kb":972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"45'94","reported_seq":380,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.452921+0000","last_change":"2024-09-18T00:57:39.382152+0000","last_active":"2024-09-18T00:58:05.452921+0000","last_peered":"2024-09-18T00:58:05.452921+0000","last_clean":"2024-09-18T00:58:05.452921+0000","last_became_active":"2024-09-18T00:57:36.904791+0000","last_became_peered":"2024-09-18T00:57:36.904791+0000","last_unstale":"2024-09-18T00:58:05.452921+0000","last_undegraded":"2024-09-18T00:58:05.452921+0000","last_fullsized":"2024-09-18T00:58:05.452921+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:15:10.983235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090629000000000003,"stat_sum":{"num_bytes":942702,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":606,"num_read_kb":738,"num_write":81,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"45'166","reported_seq":493,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.716346+0000","last_change":"2024-09-18T00:57:38.909597+0000","last_active":"2024-09-18T00:58:05.716346+0000","last_peered":"2024-09-18T00:58:05.716346+0000","last_clean":"2024-09-18T00:58:05.716346+0000","last_became_active":"2024-09-18T00:57:36.905462+0000","last_became_peered":"2024-09-18T00:57:36.905462+0000","last_unstale":"2024-09-18T00:58:05.716346+0000","last_undegraded":"2024-09-18T00:58:05.716346+0000","last_fullsized":"2024-09-18T00:58:05.716346+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:24:59.482353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081831799999999995,"stat_sum":{"num_bytes":9001370,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":799,"num_read_kb":6628,"num_write":218,"num_write_kb":4285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"45'144","reported_seq":1799,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.221446+0000","last_change":"2024-09-18T00:57:38.900906+0000","last_active":"2024-09-18T00:58:05.221446+0000","last_peered":"2024-09-18T00:58:05.221446+0000","last_clean":"2024-09-18T00:58:05.221446+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T00:58:05.221446+0000","last_undegraded":"2024-09-18T00:58:05.221446+0000","last_fullsized":"2024-09-18T00:58:05.221446+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:21:18.073028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069386899999999995,"stat_sum":{"num_bytes":30277407,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":2226,"num_read_kb":33163,"num_write":125,"num_write_kb":23122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"45'81","reported_seq":216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.643573+0000","last_change":"2024-09-18T00:57:38.899297+0000","last_active":"2024-09-18T00:58:05.643573+0000","last_peered":"2024-09-18T00:58:05.643573+0000","last_clean":"2024-09-18T00:58:05.643573+0000","last_became_active":"2024-09-18T00:57:36.897289+0000","last_became_peered":"2024-09-18T00:57:36.897289+0000","last_unstale":"2024-09-18T00:58:05.643573+0000","last_undegraded":"2024-09-18T00:58:05.643573+0000","last_fullsized":"2024-09-18T00:58:05.643573+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:04:03.352923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071487799999999995,"stat_sum":{"num_bytes":12846580,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":91,"num_read_kb":920,"num_write":64,"num_write_kb":823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"45'110","reported_seq":929,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.782935+0000","last_change":"2024-09-18T00:57:39.056761+0000","last_active":"2024-09-18T00:58:05.782935+0000","last_peered":"2024-09-18T00:58:05.782935+0000","last_clean":"2024-09-18T00:58:05.782935+0000","last_became_active":"2024-09-18T00:57:36.910991+0000","last_became_peered":"2024-09-18T00:57:36.910991+0000","last_unstale":"2024-09-18T00:58:05.782935+0000","last_undegraded":"2024-09-18T00:58:05.782935+0000","last_fullsized":"2024-09-18T00:58:05.782935+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:02:07.471326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055104700000000004,"stat_sum":{"num_bytes":11309164,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":986,"num_read_kb":9006,"num_write":130,"num_write_kb":8123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"45'194","reported_seq":666,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-18T00:58:05.280048+0000","last_change":"2024-09-18T00:57:38.899783+0000","last_active":"2024-09-18T00:58:05.280048+0000","last_peered":"2024-09-18T00:58:05.280048+0000","last_clean":"2024-09-18T00:58:05.280048+0000","last_became_active":"2024-09-18T00:57:36.904999+0000","last_became_peered":"2024-09-18T00:57:36.904999+0000","last_unstale":"2024-09-18T00:58:05.280048+0000","last_undegraded":"2024-09-18T00:58:05.280048+0000","last_fullsized":"2024-09-18T00:58:05.280048+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:29:41.433800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070399799999999995,"stat_sum":{"num_bytes":1572910,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":1322,"num_read_kb":3419,"num_write":254,"num_write_kb":1370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":342236749,"num_objects":855,"num_object_clones":134,"num_object_copies":1710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":855,"num_whiteouts":430,"num_read":37185,"num_read_kb":254658,"num_write":5301,"num_write_kb":236164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":234348544,"data_stored":247296514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4490,"ondisk_log_size":4490,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61284,"kb_used_data":25932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94310556,"statfs":{"total":96636764160,"available":96574009344,"internally_reserved":0,"allocated":26554368,"data_stored":26414261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71744,"kb_used_data":36388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94300096,"statfs":{"total":96636764160,"available":96563298304,"internally_reserved":0,"allocated":37261312,"data_stored":38449144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71596,"kb_used_data":36252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94300244,"statfs":{"total":96636764160,"available":96563449856,"internally_reserved":0,"allocated":37122048,"data_stored":38472462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64104,"kb_used_data":24600,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94307736,"statfs":{"total":96636764160,"available":96571121664,"internally_reserved":0,"allocated":25190400,"data_stored":28109470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56780,"kb_used_data":21436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94315060,"statfs":{"total":96636764160,"available":96578621440,"internally_reserved":0,"allocated":21950464,"data_stored":23380080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72484,"kb_used_data":37140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94299356,"statfs":{"total":96636764160,"available":96562540544,"internally_reserved":0,"allocated":38031360,"data_stored":39154376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77348,"kb_used_data":37840,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94294492,"statfs":{"total":96636764160,"available":96557559808,"internally_reserved":0,"allocated":38748160,"data_stored":40140790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49436,"kb_used_data":14092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322404,"statfs":{"total":96636764160,"available":96586141696,"internally_reserved":0,"allocated":14430208,"data_stored":15794271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":38277120,"data_stored":39949840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36995072,"data_stored":38408539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21467136,"data_stored":23183349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24682496,"data_stored":27900343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36773888,"data_stored":38393113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36134912,"data_stored":37623910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":26116096,"data_stored":26267499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13901824,"data_stored":15569921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:58:10.863 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T00:58:10.863 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:58:10.863 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T00:58:10.864 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T00:58:10.869 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:10.869 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:10.886 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:10.886 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:10.897 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:10.890+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:10.903 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:10.903 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:10.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:10.929 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:10.950 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:10.950 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:10.959 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:10.959 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3710 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089501223999999, 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024498702000000001, 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:10.960 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3710 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867969+0000", 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867971+0000", 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.671e-06 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867976+0000", 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7030000000000003e-06 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867981+0000", 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8520000000000003e-06 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892175+0000", 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024193538000000001 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892279+0000", 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104017 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892467+0000", 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018858699999999999 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3711 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089482904, 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024701471999999999, 2024-09-18T00:58:10.964 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3711 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867987+0000", 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6199999999999999e-07 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867989+0000", 2024-09-18T00:58:10.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867995+0000", 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.127e-06 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868000+0000", 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:10.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892500+0000", 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024499725999999999 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892587+0000", 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7232999999999996e-05 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892688+0000", 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010115000000000001 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:10.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3712 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089463945999999, 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024914385000000001, 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3712 2024-09-18T00:58:10.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:10.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868006+0000", 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7899999999999999e-07 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868008+0000", 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.84e-06 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868014+0000", 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.694e-06 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868019+0000", 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892727+0000", 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024707638000000001 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892817+0000", 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9569999999999998e-05 2024-09-18T00:58:10.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892920+0000", 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103744 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3713 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089444414999999, 2024-09-18T00:58:10.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025115858000000001, 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3713 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868025+0000", 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868026+0000", 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.58e-07 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868032+0000", 2024-09-18T00:58:10.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3790000000000003e-06 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868039+0000", 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6969999999999996e-06 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868044+0000", 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7060000000000003e-06 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:10.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.892954+0000", 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024910704999999998 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893041+0000", 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6747000000000001e-05 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893141+0000", 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010026600000000001 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:10.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3714 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089419608, 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025291557999999999, 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3714 2024-09-18T00:58:10.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868050+0000", 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868051+0000", 2024-09-18T00:58:10.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7e-07 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868052+0000", 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.863e-06 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868059+0000", 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2720000000000003e-06 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868064+0000", 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8860000000000003e-06 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893175+0000", 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025111114 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893262+0000", 2024-09-18T00:58:10.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7476000000000001e-05 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893342+0000", 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9577000000000001e-05 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3715 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089400102999999, 2024-09-18T00:58:10.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025471191000000001, 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3715 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868070+0000", 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2799999999999996e-07 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868072+0000", 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9269999999999999e-06 2024-09-18T00:58:10.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868078+0000", 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9830000000000001e-06 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868083+0000", 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.972e-06 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:10.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893373+0000", 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025290316 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893463+0000", 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9078000000000003e-05 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893541+0000", 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8386999999999993e-05 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3716 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089381045, 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025664781000000001, 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3716 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.987 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868089+0000", 2024-09-18T00:58:10.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2099999999999998e-07 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868091+0000", 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8950000000000001e-06 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868097+0000", 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1929999999999998e-06 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868102+0000", 2024-09-18T00:58:10.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6789999999999996e-06 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893570+0000", 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025468227999999999 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893657+0000", 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7176999999999996e-05 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:10.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893754+0000", 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6287999999999997e-05 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3717 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089361939, 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025815932999999999, 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:10.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3717 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868108+0000", 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868111+0000", 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6860000000000002e-06 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:10.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868117+0000", 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868122+0000", 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.69e-06 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893802+0000", 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025680397000000001 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:10.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893869+0000", 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6597e-05 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893924+0000", 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5031999999999998e-05 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3718 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089340917000001, 2024-09-18T00:58:10.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026018052, 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3718 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:10.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868129+0000", 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8700000000000001e-07 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:10.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868131+0000", 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9369999999999998e-06 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868138+0000", 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4389999999999997e-06 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868143+0000", 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1440000000000002e-06 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:10.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.893973+0000", 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025830490000000001 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894063+0000", 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9313000000000004e-05 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894147+0000", 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4342e-05 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:10.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3719 3.c 3:349a9c5e:::rbd_data.117c7eee262f.0000000000000088:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089320960999999, 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026147934000000001, 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3719 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.000 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868149+0000", 2024-09-18T00:58:11.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4499999999999998e-07 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868151+0000", 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.846e-06 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868157+0000", 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868162+0000", 2024-09-18T00:58:11.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5619999999999997e-06 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894183+0000", 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026021083 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894241+0000", 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8505999999999999e-05 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894297+0000", 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5396000000000003e-05 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3720 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089302444000001, 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026376077000000001, 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3720 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868167+0000", 2024-09-18T00:58:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868168+0000", 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868170+0000", 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0889999999999998e-06 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868176+0000", 2024-09-18T00:58:11.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3090000000000001e-06 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868181+0000", 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8810000000000004e-06 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894341+0000", 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026159636 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894430+0000", 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8986000000000005e-05 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894543+0000", 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011382700000000001 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3721 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:11.008 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089282925999999, 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569722, 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3721 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868187+0000", 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.34e-07 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868189+0000", 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7460000000000001e-06 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868195+0000", 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8490000000000002e-06 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868200+0000", 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1669999999999998e-06 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894623+0000", 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02642338 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894688+0000", 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4183000000000005e-05 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894757+0000", 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9062999999999999e-05 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.012 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3722 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089264240999999, 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026735041000000001, 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3722 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868206+0000", 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4200000000000002e-07 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868208+0000", 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8470000000000001e-06 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868214+0000", 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1090000000000003e-06 2024-09-18T00:58:11.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868218+0000", 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.476e-06 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894788+0000", 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026569274 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894868+0000", 2024-09-18T00:58:11.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0711999999999996e-05 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.894941+0000", 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3723 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089245824999999, 2024-09-18T00:58:11.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026971922999999998, 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3723 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868224+0000", 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-07 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868226+0000", 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.812e-06 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868232+0000", 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7910000000000001e-06 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868237+0000", 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7339999999999999e-06 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895056+0000", 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026819195000000001 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895103+0000", 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7264e-05 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895196+0000", 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2802999999999999e-05 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3724 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:11.021 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089227612999998, 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027181833999999998, 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3724 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868242+0000", 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868243+0000", 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6399999999999998e-07 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868245+0000", 2024-09-18T00:58:11.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1160000000000002e-06 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868251+0000", 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1410000000000002e-06 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868256+0000", 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0130000000000001e-06 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895249+0000", 2024-09-18T00:58:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02699352 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895338+0000", 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8331000000000005e-05 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895424+0000", 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6348999999999996e-05 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3725 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.089208479, 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027296051000000002, 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3725 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868261+0000", 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868262+0000", 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4900000000000001e-07 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868267+0000", 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7259999999999999e-06 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868274+0000", 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2360000000000001e-06 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.868278+0000", 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7620000000000003e-06 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895458+0000", 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027179608000000001 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895511+0000", 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2738000000000002e-05 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895557+0000", 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6631999999999997e-05 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.029 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3727 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.085897928000001, 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024398075000000002, 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.030 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3727 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871572+0000", 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871573+0000", 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.367e-06 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871580+0000", 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8029999999999999e-06 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871600+0000", 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0213000000000001e-05 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871615+0000", 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.471e-05 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895805+0000", 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024189677 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895923+0000", 2024-09-18T00:58:11.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118496 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.895970+0000", 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6808999999999999e-05 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3728 3.c 3:37a5289f:::rbd_data.117c7eee262f.0000000000000089:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.085836668999999, 2024-09-18T00:58:11.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024637071999999999, 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3728 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871633+0000", 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871634+0000", 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.074e-06 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871642+0000", 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8830000000000006e-06 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871660+0000", 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8283000000000001e-05 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.871692+0000", 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1714000000000002e-05 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896004+0000", 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.024312099 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896099+0000", 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5036000000000003e-05 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.896270+0000", 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017098300000000001 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.038 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.89350481, 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:58:11.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:58:11.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15723 3.f 3:f7b50ba2:::rbd_data.117c53e529c1.0000000000000058:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.735235884000002, 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026056645999999999, 2024-09-18T00:58:11.044 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15723 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222234+0000", 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222235+0000", 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222242+0000", 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4710000000000004e-06 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222255+0000", 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2877999999999999e-05 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222261+0000", 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.489e-06 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222328+0000", 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7280999999999995e-05 2024-09-18T00:58:11.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.222502+0000", 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000173609 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.227120+0000", 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046180129999999998 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248104+0000", 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020983979 2024-09-18T00:58:11.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248166+0000", 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2340000000000003e-05 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248236+0000", 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9227000000000007e-05 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:53.248291+0000", 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5019000000000003e-05 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.049 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.050 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.050 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:11.050 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:11.051 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:11.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:11.052 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:11.053 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:11.053 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:11.046+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:11.116 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:58:11.151 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:11.146+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 51, 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "82a96d76-ba83-49e2-a985-84604bd599ba", 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-18T00:57:07.138732+0000", 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-18T00:58:11.106699+0000", 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-18T00:57:20.258605+0000", 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-18T00:57:09.756380+0000", 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5799936, 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-18T00:58:11.173 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-18T00:58:11.174 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "jewel", 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:21.962521+0000", 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:58:11.175 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-18T00:58:11.176 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "19", 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:58:11.178 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:58:11.179 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-18T00:58:11.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 8, 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 8, 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.25, 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.181 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:25.721733+0000", 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:58:11.182 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:58:11.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "23", 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 23, 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:58:11.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:58:11.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T00:58:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 4.9800000190734863, 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 4.9800000190734863, 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.87999999523162842, 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 4.380000114440918, 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 4.380000114440918, 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:58:11.187 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:35.599413+0000", 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:58:11.188 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:58:11.189 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "51", 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 18, 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 51, 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:58:11.190 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:58:11.191 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:58:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 1.75, 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 1.75, 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 1, 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.75, 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.75, 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:58:11.193 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "3f359ce5-6f6e-4161-8554-8ac028347274", 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:58:11.194 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 24, 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:58:11.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/199763443", 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/199763443", 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/199763443", 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/199763443", 2024-09-18T00:58:11.199 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "48687611-1556-4b6d-86af-7872494a91d7", 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:58:11.200 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 24, 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.201 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.202 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/345149800", 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/345149800", 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/345149800", 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/345149800", 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:58:11.205 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "9e6b1f90-ce9b-46d4-92fe-d457f7e415d5", 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 24, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:58:11.206 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.207 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:58:11.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:58:11.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/431239931", 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/431239931", 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/431239931", 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/431239931", 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5", 2024-09-18T00:58:11.211 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 24, 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.212 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/1775474180", 2024-09-18T00:58:11.216 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/1775474180", 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/1775474180", 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/1775474180", 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "802ccbeb-7351-4b38-832a-695117c7e493", 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:58:11.217 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 24, 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6808", 2024-09-18T00:58:11.218 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6809", 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6810", 2024-09-18T00:58:11.219 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6811", 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6814", 2024-09-18T00:58:11.220 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6815", 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6812", 2024-09-18T00:58:11.221 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6813", 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6809/4198401207", 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6811/4198401207", 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6815/4198401207", 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6813/4198401207", 2024-09-18T00:58:11.222 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d958b3da-efa1-4d86-b81d-e78874441437", 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:58:11.223 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 24, 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6824", 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6825", 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6826", 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6827", 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6830", 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6831", 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6828", 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6829", 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6825/2599825823", 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6827/2599825823", 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6831/2599825823", 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6829/2599825823", 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:58:11.228 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "51b3cea8-c686-4fd3-b3f4-ba936cf26b64", 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 24, 2024-09-18T00:58:11.229 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6800", 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6801", 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:58:11.230 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6802", 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6803", 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:58:11.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6806", 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6807", 2024-09-18T00:58:11.232 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6804", 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.233 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6805", 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6801/3550656451", 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6803/3550656451", 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6807/3550656451", 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6805/3550656451", 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b02ead1c-837a-4b07-9ad1-4e903ed5bc4e", 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 24, 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:58:11.235 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6816", 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6817", 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6818", 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6819", 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.237 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6822", 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6823", 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.238 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6820", 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6821", 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.239 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6817/2993352359", 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6819/2993352359", 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6823/2993352359", 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6821/2993352359", 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:58:11.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.896477+0000", 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:58:11.241 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.941756+0000", 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:58:11.242 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.560180+0000", 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.077394+0000", 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.996264+0000", 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:58:11.244 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.511686+0000", 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:58:11.245 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.411051+0000", 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.888092+0000", 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:58:11.246 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-18T00:58:11.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [], 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-18T00:58:11.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.249 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:11.250 INFO:tasks.thrashosds.thrasher:j is {'epoch': 51, 'fsid': '82a96d76-ba83-49e2-a985-84604bd599ba', 'created': '2024-09-18T00:57:07.138732+0000', 'modified': '2024-09-18T00:58:11.106699+0000', 'last_up_change': '2024-09-18T00:57:20.258605+0000', 'last_in_change': '2024-09-18T00:57:09.756380+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T00:57:21.962521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T00:57:25.721733+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T00:57:35.599413+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 18, 'snap_epoch': 51, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3f359ce5-6f6e-4161-8554-8ac028347274', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 199763443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 199763443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 199763443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 199763443}]}, 'public_addr': '172.21.15.42:6801/199763443', 'cluster_addr': '172.21.15.42:6803/199763443', 'heartbeat_back_addr': '172.21.15.42:6807/199763443', 'heartbeat_front_addr': '172.21.15.42:6805/199763443', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '48687611-1556-4b6d-86af-7872494a91d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 345149800}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 345149800}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 345149800}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 345149800}]}, 'public_addr': '172.21.15.42:6809/345149800', 'cluster_addr': '172.21.15.42:6811/345149800', 'heartbeat_back_addr': '172.21.15.42:6815/345149800', 'heartbeat_front_addr': '172.21.15.42:6813/345149800', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9e6b1f90-ce9b-46d4-92fe-d457f7e415d5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 431239931}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 431239931}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 431239931}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 431239931}]}, 'public_addr': '172.21.15.42:6825/431239931', 'cluster_addr': '172.21.15.42:6827/431239931', 'heartbeat_back_addr': '172.21.15.42:6831/431239931', 'heartbeat_front_addr': '172.21.15.42:6829/431239931', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 1775474180}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 1775474180}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 1775474180}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 1775474180}]}, 'public_addr': '172.21.15.42:6817/1775474180', 'cluster_addr': '172.21.15.42:6819/1775474180', 'heartbeat_back_addr': '172.21.15.42:6823/1775474180', 'heartbeat_front_addr': '172.21.15.42:6821/1775474180', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '802ccbeb-7351-4b38-832a-695117c7e493', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6809', 'nonce': 4198401207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6811', 'nonce': 4198401207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6815', 'nonce': 4198401207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6813', 'nonce': 4198401207}]}, 'public_addr': '172.21.15.195:6809/4198401207', 'cluster_addr': '172.21.15.195:6811/4198401207', 'heartbeat_back_addr': '172.21.15.195:6815/4198401207', 'heartbeat_front_addr': '172.21.15.195:6813/4198401207', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd958b3da-efa1-4d86-b81d-e78874441437', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6824', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6825', 'nonce': 2599825823}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6826', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6827', 'nonce': 2599825823}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6830', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6831', 'nonce': 2599825823}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6828', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6829', 'nonce': 2599825823}]}, 'public_addr': '172.21.15.195:6825/2599825823', 'cluster_addr': '172.21.15.195:6827/2599825823', 'heartbeat_back_addr': '172.21.15.195:6831/2599825823', 'heartbeat_front_addr': '172.21.15.195:6829/2599825823', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '51b3cea8-c686-4fd3-b3f4-ba936cf26b64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6800', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6801', 'nonce': 3550656451}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6803', 'nonce': 3550656451}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6807', 'nonce': 3550656451}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6805', 'nonce': 3550656451}]}, 'public_addr': '172.21.15.195:6801/3550656451', 'cluster_addr': '172.21.15.195:6803/3550656451', 'heartbeat_back_addr': '172.21.15.195:6807/3550656451', 'heartbeat_front_addr': '172.21.15.195:6805/3550656451', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b02ead1c-837a-4b07-9ad1-4e903ed5bc4e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6817', 'nonce': 2993352359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6818', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6819', 'nonce': 2993352359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6822', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6823', 'nonce': 2993352359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6820', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6821', 'nonce': 2993352359}]}, 'public_addr': '172.21.15.195:6817/2993352359', 'cluster_addr': '172.21.15.195:6819/2993352359', 'heartbeat_back_addr': '172.21.15.195:6823/2993352359', 'heartbeat_front_addr': '172.21.15.195:6821/2993352359', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.896477+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.941756+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.560180+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.077394+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.996264+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.511686+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.411051+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.888092+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T00:58:11.251 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:58:11.251 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:11.251 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:11.251 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:11.251 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:11.251 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone15 intact 2024-09-18T00:58:11.253 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:11.246+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:11.259 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:11.353 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:11.346+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:11.432 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:11.432 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:11.432 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:11.450 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:11.454 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-18T00:58:11.455 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:11.450+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:11.471 INFO:teuthology.orchestra.run.smithi042.stdout:1614 punch from 0x43c5356 to 0x43d1cd5, (0xc97f bytes) 2024-09-18T00:58:11.477 INFO:teuthology.orchestra.run.smithi042.stdout:1615 write 0x5f1735c thru 0x5f195c0 (0x2265 bytes) 2024-09-18T00:58:11.556 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:11.550+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:11.595 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 48.303352357000001, 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:11.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:58:11.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7423 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.107053807, 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045296661000000002, 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7423 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485621+0000", 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485627+0000", 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.156e-06 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485633+0000", 2024-09-18T00:58:11.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789999999999999e-06 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530788+0000", 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045155001 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530853+0000", 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5634000000000002e-05 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530915+0000", 2024-09-18T00:58:11.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2022999999999999e-05 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7424 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1146880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.107032817, 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045419355000000002, 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7424 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485642+0000", 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0640000000000001e-06 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485649+0000", 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485654+0000", 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2329999999999997e-06 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530941+0000", 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045286602000000002 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531004+0000", 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3455999999999997e-05 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531059+0000", 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4771000000000001e-05 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.611 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.107011304, 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:11.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:58:11.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.106989866999999, 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:58:11.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:58:11.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.106894539999999, 2024-09-18T00:58:11.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:58:11.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.106869101000001, 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:58:11.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.106846577999999, 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.106823137999999, 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:11.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:58:11.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:58:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:58:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:58:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.638 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.106799328000001, 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:11.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.642 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.106776963000002, 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.106750345999998, 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.651 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.102598574999998, 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:58:11.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:58:11.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.102577832000001, 2024-09-18T00:58:11.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:58:11.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.102532267000001, 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:58:11.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:11.665 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.102505925999999, 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:58:11.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.669 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.102483255999999, 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.102464883, 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.678 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.102445095, 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:58:11.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:58:11.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.102418225000001, 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:11.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:11.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:11.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:11.687 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:11.689 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:11.689 INFO:teuthology.orchestra.run.smithi042.stdout:1616 write 0x492748a thru 0x492ac2f (0x37a6 bytes) 2024-09-18T00:58:11.690 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:58:11.691 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:58:11.692 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-18T00:58:11.109258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":410610266,"num_objects":997,"num_object_clones":187,"num_object_copies":1994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":997,"num_whiteouts":486,"num_read":40453,"num_read_kb":264352,"num_write":6183,"num_write_kb":256806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5266,"ondisk_log_size":5266,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":553048,"kb_used_data":257732,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754421672,"statfs":{"total":773094113280,"available":772527792128,"internally_reserved":0,"allocated":263917568,"data_stored":274010347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":302173744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":67914202,"num_objects":142,"num_object_clones":54,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":57,"num_read":3215,"num_read_kb":9898,"num_write":729,"num_write_kb":18244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.043281"},"pg_stats":[{"pgid":"3.1f","version":"49'155","reported_seq":985,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.125179+0000","last_change":"2024-09-18T00:57:38.909849+0000","last_active":"2024-09-18T00:58:10.125179+0000","last_peered":"2024-09-18T00:58:10.125179+0000","last_clean":"2024-09-18T00:58:10.125179+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T00:58:10.125179+0000","last_undegraded":"2024-09-18T00:58:10.125179+0000","last_fullsized":"2024-09-18T00:58:10.125179+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:38.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001565786,"stat_sum":{"num_bytes":3571728,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":15,"num_read":1775,"num_read_kb":4913,"num_write":139,"num_write_kb":2366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"48'88","reported_seq":847,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.125295+0000","last_change":"2024-09-18T00:57:38.908434+0000","last_active":"2024-09-18T00:58:10.125295+0000","last_peered":"2024-09-18T00:58:10.125295+0000","last_clean":"2024-09-18T00:58:10.125295+0000","last_became_active":"2024-09-18T00:57:36.897948+0000","last_became_peered":"2024-09-18T00:57:36.897948+0000","last_unstale":"2024-09-18T00:58:10.125295+0000","last_undegraded":"2024-09-18T00:58:10.125295+0000","last_fullsized":"2024-09-18T00:58:10.125295+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:33:42.064526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040574399999999998,"stat_sum":{"num_bytes":5035999,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":19,"num_read":730,"num_read_kb":4754,"num_write":56,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"48'306","reported_seq":1006,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.100521+0000","last_change":"2024-09-18T00:57:38.902294+0000","last_active":"2024-09-18T00:58:10.100521+0000","last_peered":"2024-09-18T00:58:10.100521+0000","last_clean":"2024-09-18T00:58:10.100521+0000","last_became_active":"2024-09-18T00:57:36.906790+0000","last_became_peered":"2024-09-18T00:57:36.906790+0000","last_unstale":"2024-09-18T00:58:10.100521+0000","last_undegraded":"2024-09-18T00:58:10.100521+0000","last_fullsized":"2024-09-18T00:58:10.100521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:39:44.501982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039089,"stat_sum":{"num_bytes":10096338,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":1775,"num_read_kb":12064,"num_write":461,"num_write_kb":1754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"48'291","reported_seq":1261,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.127665+0000","last_change":"2024-09-18T00:57:38.901600+0000","last_active":"2024-09-18T00:58:10.127665+0000","last_peered":"2024-09-18T00:58:10.127665+0000","last_clean":"2024-09-18T00:58:10.127665+0000","last_became_active":"2024-09-18T00:57:36.906003+0000","last_became_peered":"2024-09-18T00:57:36.906003+0000","last_unstale":"2024-09-18T00:58:10.127665+0000","last_undegraded":"2024-09-18T00:58:10.127665+0000","last_fullsized":"2024-09-18T00:58:10.127665+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:58:04.664160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000711266,"stat_sum":{"num_bytes":4312182,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":14,"num_read":2267,"num_read_kb":8421,"num_write":355,"num_write_kb":1567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"50'320","reported_seq":1786,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.127391+0000","last_change":"2024-09-18T00:57:38.921854+0000","last_active":"2024-09-18T00:58:10.127391+0000","last_peered":"2024-09-18T00:58:10.127391+0000","last_clean":"2024-09-18T00:58:10.127391+0000","last_became_active":"2024-09-18T00:57:36.913393+0000","last_became_peered":"2024-09-18T00:57:36.913393+0000","last_unstale":"2024-09-18T00:58:10.127391+0000","last_undegraded":"2024-09-18T00:58:10.127391+0000","last_fullsized":"2024-09-18T00:58:10.127391+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:14:51.299491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002554022,"stat_sum":{"num_bytes":24324905,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":2423,"num_read_kb":9655,"num_write":463,"num_write_kb":11806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"49'138","reported_seq":1392,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.158901+0000","last_change":"2024-09-18T00:57:38.901805+0000","last_active":"2024-09-18T00:58:10.158901+0000","last_peered":"2024-09-18T00:58:10.158901+0000","last_clean":"2024-09-18T00:58:10.158901+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T00:58:10.158901+0000","last_undegraded":"2024-09-18T00:58:10.158901+0000","last_fullsized":"2024-09-18T00:58:10.158901+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:27:20.903711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098070899999999996,"stat_sum":{"num_bytes":26226007,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":1948,"num_read_kb":7762,"num_write":113,"num_write_kb":674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"50'138","reported_seq":1923,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.184993+0000","last_change":"2024-09-18T00:57:38.922010+0000","last_active":"2024-09-18T00:58:10.184993+0000","last_peered":"2024-09-18T00:58:10.184993+0000","last_clean":"2024-09-18T00:58:10.184993+0000","last_became_active":"2024-09-18T00:57:36.910457+0000","last_became_peered":"2024-09-18T00:57:36.910457+0000","last_unstale":"2024-09-18T00:58:10.184993+0000","last_undegraded":"2024-09-18T00:58:10.184993+0000","last_fullsized":"2024-09-18T00:58:10.184993+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:58.232847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026842849999999998,"stat_sum":{"num_bytes":28518828,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":1781,"num_read_kb":9568,"num_write":137,"num_write_kb":13082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"48'168","reported_seq":1036,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.126527+0000","last_change":"2024-09-18T00:57:38.908144+0000","last_active":"2024-09-18T00:58:10.126527+0000","last_peered":"2024-09-18T00:58:10.126527+0000","last_clean":"2024-09-18T00:58:10.126527+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T00:58:10.126527+0000","last_undegraded":"2024-09-18T00:58:10.126527+0000","last_fullsized":"2024-09-18T00:58:10.126527+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:07.734668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000871597,"stat_sum":{"num_bytes":28595971,"num_objects":29,"num_object_clones":3,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":1435,"num_read_kb":1499,"num_write":177,"num_write_kb":33827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"48'113","reported_seq":670,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.125443+0000","last_change":"2024-09-18T00:57:38.910038+0000","last_active":"2024-09-18T00:58:10.125443+0000","last_peered":"2024-09-18T00:58:10.125443+0000","last_clean":"2024-09-18T00:58:10.125443+0000","last_became_active":"2024-09-18T00:57:36.895915+0000","last_became_peered":"2024-09-18T00:57:36.895915+0000","last_unstale":"2024-09-18T00:58:10.125443+0000","last_undegraded":"2024-09-18T00:58:10.125443+0000","last_fullsized":"2024-09-18T00:58:10.125443+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:07.638522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001269769,"stat_sum":{"num_bytes":8396709,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":814,"num_read_kb":11695,"num_write":96,"num_write_kb":4072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"48'181","reported_seq":2442,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.102178+0000","last_change":"2024-09-18T00:57:39.381845+0000","last_active":"2024-09-18T00:58:10.102178+0000","last_peered":"2024-09-18T00:58:10.102178+0000","last_clean":"2024-09-18T00:58:10.102178+0000","last_became_active":"2024-09-18T00:57:36.894538+0000","last_became_peered":"2024-09-18T00:57:36.894538+0000","last_unstale":"2024-09-18T00:58:10.102178+0000","last_undegraded":"2024-09-18T00:58:10.102178+0000","last_fullsized":"2024-09-18T00:58:10.102178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:57:44.029150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076552,"stat_sum":{"num_bytes":7854719,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":2775,"num_read_kb":12322,"num_write":313,"num_write_kb":15012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"48'112","reported_seq":604,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.124349+0000","last_change":"2024-09-18T00:57:38.909647+0000","last_active":"2024-09-18T00:58:10.124349+0000","last_peered":"2024-09-18T00:58:10.124349+0000","last_clean":"2024-09-18T00:58:10.124349+0000","last_became_active":"2024-09-18T00:57:36.902977+0000","last_became_peered":"2024-09-18T00:57:36.902977+0000","last_unstale":"2024-09-18T00:58:10.124349+0000","last_undegraded":"2024-09-18T00:58:10.124349+0000","last_fullsized":"2024-09-18T00:58:10.124349+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:08:57.447261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051351900000000004,"stat_sum":{"num_bytes":23482430,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":17,"num_read":447,"num_read_kb":7407,"num_write":140,"num_write_kb":38140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"48'44","reported_seq":147,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.100419+0000","last_change":"2024-09-18T00:57:38.902578+0000","last_active":"2024-09-18T00:58:10.100419+0000","last_peered":"2024-09-18T00:58:10.100419+0000","last_clean":"2024-09-18T00:58:10.100419+0000","last_became_active":"2024-09-18T00:57:36.894977+0000","last_became_peered":"2024-09-18T00:57:36.894977+0000","last_unstale":"2024-09-18T00:58:10.100419+0000","last_undegraded":"2024-09-18T00:58:10.100419+0000","last_fullsized":"2024-09-18T00:58:10.100419+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:16.301854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028223319999999999,"stat_sum":{"num_bytes":787963,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":8,"num_read":46,"num_read_kb":35,"num_write":35,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.101348+0000","last_change":"2024-09-18T00:57:29.328163+0000","last_active":"2024-09-18T00:58:10.101348+0000","last_peered":"2024-09-18T00:58:10.101348+0000","last_clean":"2024-09-18T00:58:10.101348+0000","last_became_active":"2024-09-18T00:57:27.311231+0000","last_became_peered":"2024-09-18T00:57:27.311231+0000","last_unstale":"2024-09-18T00:58:10.101348+0000","last_undegraded":"2024-09-18T00:58:10.101348+0000","last_fullsized":"2024-09-18T00:58:10.101348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:57.340012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020990940000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"48'191","reported_seq":1326,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.124768+0000","last_change":"2024-09-18T00:57:38.920098+0000","last_active":"2024-09-18T00:58:10.124768+0000","last_peered":"2024-09-18T00:58:10.124768+0000","last_clean":"2024-09-18T00:58:10.124768+0000","last_became_active":"2024-09-18T00:57:36.920599+0000","last_became_peered":"2024-09-18T00:57:36.920599+0000","last_unstale":"2024-09-18T00:58:10.124768+0000","last_undegraded":"2024-09-18T00:58:10.124768+0000","last_fullsized":"2024-09-18T00:58:10.124768+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:57.700337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276634,"stat_sum":{"num_bytes":10760753,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":1683,"num_read_kb":14337,"num_write":260,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.124895+0000","last_change":"2024-09-18T00:57:29.327841+0000","last_active":"2024-09-18T00:58:10.124895+0000","last_peered":"2024-09-18T00:58:10.124895+0000","last_clean":"2024-09-18T00:58:10.124895+0000","last_became_active":"2024-09-18T00:57:27.315417+0000","last_became_peered":"2024-09-18T00:57:27.315417+0000","last_unstale":"2024-09-18T00:58:10.124895+0000","last_undegraded":"2024-09-18T00:58:10.124895+0000","last_fullsized":"2024-09-18T00:58:10.124895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:54.335334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'141","reported_seq":716,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.143889+0000","last_change":"2024-09-18T00:57:38.920634+0000","last_active":"2024-09-18T00:58:10.143889+0000","last_peered":"2024-09-18T00:58:10.143889+0000","last_clean":"2024-09-18T00:58:10.143889+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T00:58:10.143889+0000","last_undegraded":"2024-09-18T00:58:10.143889+0000","last_fullsized":"2024-09-18T00:58:10.143889+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:07:57.269130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728301,"stat_sum":{"num_bytes":13194079,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":17,"num_read":1097,"num_read_kb":1683,"num_write":113,"num_write_kb":1891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.100771+0000","last_change":"2024-09-18T00:57:29.327995+0000","last_active":"2024-09-18T00:58:10.100771+0000","last_peered":"2024-09-18T00:58:10.100771+0000","last_clean":"2024-09-18T00:58:10.100771+0000","last_became_active":"2024-09-18T00:57:27.316238+0000","last_became_peered":"2024-09-18T00:57:27.316238+0000","last_unstale":"2024-09-18T00:58:10.100771+0000","last_undegraded":"2024-09-18T00:58:10.100771+0000","last_fullsized":"2024-09-18T00:58:10.100771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019441149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"48'87","reported_seq":471,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.099381+0000","last_change":"2024-09-18T00:57:39.056519+0000","last_active":"2024-09-18T00:58:10.099381+0000","last_peered":"2024-09-18T00:58:10.099381+0000","last_clean":"2024-09-18T00:58:10.099381+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T00:58:10.099381+0000","last_undegraded":"2024-09-18T00:58:10.099381+0000","last_fullsized":"2024-09-18T00:58:10.099381+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:40.862178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043106399999999999,"stat_sum":{"num_bytes":11830341,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":16,"num_read":346,"num_read_kb":3333,"num_write":75,"num_write_kb":7929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.125151+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:58:10.125151+0000","last_peered":"2024-09-18T00:58:10.125151+0000","last_clean":"2024-09-18T00:58:10.125151+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:58:10.125151+0000","last_undegraded":"2024-09-18T00:58:10.125151+0000","last_fullsized":"2024-09-18T00:58:10.125151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"48'83","reported_seq":635,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.101983+0000","last_change":"2024-09-18T00:57:39.381374+0000","last_active":"2024-09-18T00:58:10.101983+0000","last_peered":"2024-09-18T00:58:10.101983+0000","last_clean":"2024-09-18T00:58:10.101983+0000","last_became_active":"2024-09-18T00:57:36.902781+0000","last_became_peered":"2024-09-18T00:57:36.902781+0000","last_unstale":"2024-09-18T00:58:10.101983+0000","last_undegraded":"2024-09-18T00:58:10.101983+0000","last_fullsized":"2024-09-18T00:58:10.101983+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:02:51.032920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000584944,"stat_sum":{"num_bytes":3586770,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":15,"num_read":785,"num_read_kb":9661,"num_write":72,"num_write_kb":686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"46'210","reported_seq":1092,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.124329+0000","last_change":"2024-09-18T00:57:38.909273+0000","last_active":"2024-09-18T00:58:10.124329+0000","last_peered":"2024-09-18T00:58:10.124329+0000","last_clean":"2024-09-18T00:58:10.124329+0000","last_became_active":"2024-09-18T00:57:36.906264+0000","last_became_peered":"2024-09-18T00:57:36.906264+0000","last_unstale":"2024-09-18T00:58:10.124329+0000","last_undegraded":"2024-09-18T00:58:10.124329+0000","last_fullsized":"2024-09-18T00:58:10.124329+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:33:53.000602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056105599999999999,"stat_sum":{"num_bytes":25027841,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":1546,"num_read_kb":1693,"num_write":306,"num_write_kb":15813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":75,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.100308+0000","last_change":"2024-09-18T00:57:29.327436+0000","last_active":"2024-09-18T00:58:10.100308+0000","last_peered":"2024-09-18T00:58:10.100308+0000","last_clean":"2024-09-18T00:58:10.100308+0000","last_became_active":"2024-09-18T00:57:27.410600+0000","last_became_peered":"2024-09-18T00:57:27.410600+0000","last_unstale":"2024-09-18T00:58:10.100308+0000","last_undegraded":"2024-09-18T00:58:10.100308+0000","last_fullsized":"2024-09-18T00:58:10.100308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:59.380173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"50'207","reported_seq":532,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.129534+0000","last_change":"2024-09-18T00:57:38.907753+0000","last_active":"2024-09-18T00:58:10.129534+0000","last_peered":"2024-09-18T00:58:10.129534+0000","last_clean":"2024-09-18T00:58:10.129534+0000","last_became_active":"2024-09-18T00:57:36.907660+0000","last_became_peered":"2024-09-18T00:57:36.907660+0000","last_unstale":"2024-09-18T00:58:10.129534+0000","last_undegraded":"2024-09-18T00:58:10.129534+0000","last_fullsized":"2024-09-18T00:58:10.129534+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:33.279969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090282999999999997,"stat_sum":{"num_bytes":14224233,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":805,"num_read_kb":4692,"num_write":264,"num_write_kb":3487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":69,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T00:58:08.077331+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:58:08.077331+0000","last_peered":"2024-09-18T00:58:08.077331+0000","last_clean":"2024-09-18T00:58:08.077331+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:58:08.077331+0000","last_undegraded":"2024-09-18T00:58:08.077331+0000","last_fullsized":"2024-09-18T00:58:08.077331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"48'85","reported_seq":860,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.099347+0000","last_change":"2024-09-18T00:57:39.056604+0000","last_active":"2024-09-18T00:58:10.099347+0000","last_peered":"2024-09-18T00:58:10.099347+0000","last_clean":"2024-09-18T00:58:10.099347+0000","last_became_active":"2024-09-18T00:57:36.903081+0000","last_became_peered":"2024-09-18T00:57:36.903081+0000","last_unstale":"2024-09-18T00:58:10.099347+0000","last_undegraded":"2024-09-18T00:58:10.099347+0000","last_fullsized":"2024-09-18T00:58:10.099347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:44:28.623617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045206800000000002,"stat_sum":{"num_bytes":9486939,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":737,"num_read_kb":11698,"num_write":68,"num_write_kb":883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":73,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.099919+0000","last_change":"2024-09-18T00:57:29.327276+0000","last_active":"2024-09-18T00:58:10.099919+0000","last_peered":"2024-09-18T00:58:10.099919+0000","last_clean":"2024-09-18T00:58:10.099919+0000","last_became_active":"2024-09-18T00:57:27.316217+0000","last_became_peered":"2024-09-18T00:57:27.316217+0000","last_unstale":"2024-09-18T00:58:10.099919+0000","last_undegraded":"2024-09-18T00:58:10.099919+0000","last_fullsized":"2024-09-18T00:58:10.099919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:38.026720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012774469999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"48'109","reported_seq":355,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.124626+0000","last_change":"2024-09-18T00:57:38.919998+0000","last_active":"2024-09-18T00:58:10.124626+0000","last_peered":"2024-09-18T00:58:10.124626+0000","last_clean":"2024-09-18T00:58:10.124626+0000","last_became_active":"2024-09-18T00:57:36.906680+0000","last_became_peered":"2024-09-18T00:57:36.906680+0000","last_unstale":"2024-09-18T00:58:10.124626+0000","last_undegraded":"2024-09-18T00:58:10.124626+0000","last_fullsized":"2024-09-18T00:58:10.124626+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:01:18.666746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064609,"stat_sum":{"num_bytes":13997069,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":216,"num_read_kb":2211,"num_write":95,"num_write_kb":33815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.100639+0000","last_change":"2024-09-18T00:57:29.328733+0000","last_active":"2024-09-18T00:58:10.100639+0000","last_peered":"2024-09-18T00:58:10.100639+0000","last_clean":"2024-09-18T00:58:10.100639+0000","last_became_active":"2024-09-18T00:57:27.410396+0000","last_became_peered":"2024-09-18T00:57:27.410396+0000","last_unstale":"2024-09-18T00:58:10.100639+0000","last_undegraded":"2024-09-18T00:58:10.100639+0000","last_fullsized":"2024-09-18T00:58:10.100639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:27.438117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":224,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.127448+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:58:10.127448+0000","last_peered":"2024-09-18T00:58:10.127448+0000","last_clean":"2024-09-18T00:58:10.127448+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:58:10.127448+0000","last_undegraded":"2024-09-18T00:58:10.127448+0000","last_fullsized":"2024-09-18T00:58:10.127448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"48'84","reported_seq":749,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.124540+0000","last_change":"2024-09-18T00:57:38.919534+0000","last_active":"2024-09-18T00:58:10.124540+0000","last_peered":"2024-09-18T00:58:10.124540+0000","last_clean":"2024-09-18T00:58:10.124540+0000","last_became_active":"2024-09-18T00:57:36.908955+0000","last_became_peered":"2024-09-18T00:57:36.908955+0000","last_unstale":"2024-09-18T00:58:10.124540+0000","last_undegraded":"2024-09-18T00:58:10.124540+0000","last_fullsized":"2024-09-18T00:58:10.124540+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:35:22.727595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077760500000000001,"stat_sum":{"num_bytes":15125012,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":700,"num_read_kb":14159,"num_write":92,"num_write_kb":11286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"44'82","reported_seq":595,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.100036+0000","last_change":"2024-09-18T00:57:38.899513+0000","last_active":"2024-09-18T00:58:10.100036+0000","last_peered":"2024-09-18T00:58:10.100036+0000","last_clean":"2024-09-18T00:58:10.100036+0000","last_became_active":"2024-09-18T00:57:36.911337+0000","last_became_peered":"2024-09-18T00:57:36.911337+0000","last_unstale":"2024-09-18T00:58:10.100036+0000","last_undegraded":"2024-09-18T00:58:10.100036+0000","last_fullsized":"2024-09-18T00:58:10.100036+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:33.100681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061570699999999995,"stat_sum":{"num_bytes":9928288,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":12,"num_read":464,"num_read_kb":2983,"num_write":68,"num_write_kb":7916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"48'113","reported_seq":800,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.099980+0000","last_change":"2024-09-18T00:57:38.899904+0000","last_active":"2024-09-18T00:58:10.099980+0000","last_peered":"2024-09-18T00:58:10.099980+0000","last_clean":"2024-09-18T00:58:10.099980+0000","last_became_active":"2024-09-18T00:57:36.901837+0000","last_became_peered":"2024-09-18T00:57:36.901837+0000","last_unstale":"2024-09-18T00:58:10.099980+0000","last_undegraded":"2024-09-18T00:58:10.099980+0000","last_fullsized":"2024-09-18T00:58:10.099980+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:19:07.642469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097047300000000002,"stat_sum":{"num_bytes":7888341,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":671,"num_read_kb":11952,"num_write":87,"num_write_kb":926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"48'99","reported_seq":1542,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T00:58:08.646238+0000","last_change":"2024-09-18T00:57:38.899763+0000","last_active":"2024-09-18T00:58:08.646238+0000","last_peered":"2024-09-18T00:58:08.646238+0000","last_clean":"2024-09-18T00:58:08.646238+0000","last_became_active":"2024-09-18T00:57:36.905869+0000","last_became_peered":"2024-09-18T00:57:36.905869+0000","last_unstale":"2024-09-18T00:58:08.646238+0000","last_undegraded":"2024-09-18T00:58:08.646238+0000","last_fullsized":"2024-09-18T00:58:08.646238+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:13.804442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097615600000000005,"stat_sum":{"num_bytes":11850172,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":1601,"num_read_kb":21540,"num_write":70,"num_write_kb":1353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"50'205","reported_seq":658,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.129548+0000","last_change":"2024-09-18T00:57:38.909165+0000","last_active":"2024-09-18T00:58:10.129548+0000","last_peered":"2024-09-18T00:58:10.129548+0000","last_clean":"2024-09-18T00:58:10.129548+0000","last_became_active":"2024-09-18T00:57:36.908288+0000","last_became_peered":"2024-09-18T00:57:36.908288+0000","last_unstale":"2024-09-18T00:58:10.129548+0000","last_undegraded":"2024-09-18T00:58:10.129548+0000","last_fullsized":"2024-09-18T00:58:10.129548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:04:56.894736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044375399999999998,"stat_sum":{"num_bytes":9790252,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":999,"num_read_kb":7498,"num_write":286,"num_write_kb":1416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"48'435","reported_seq":1409,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.101632+0000","last_change":"2024-09-18T00:57:38.903046+0000","last_active":"2024-09-18T00:58:10.101632+0000","last_peered":"2024-09-18T00:58:10.101632+0000","last_clean":"2024-09-18T00:58:10.101632+0000","last_became_active":"2024-09-18T00:57:36.900286+0000","last_became_peered":"2024-09-18T00:57:36.900286+0000","last_unstale":"2024-09-18T00:58:10.101632+0000","last_undegraded":"2024-09-18T00:58:10.101632+0000","last_fullsized":"2024-09-18T00:58:10.101632+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:21.732617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023201229999999999,"stat_sum":{"num_bytes":7157911,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":18,"num_read":3415,"num_read_kb":10346,"num_write":619,"num_write_kb":3129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"48'105","reported_seq":408,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.102277+0000","last_change":"2024-09-18T00:57:39.382152+0000","last_active":"2024-09-18T00:58:10.102277+0000","last_peered":"2024-09-18T00:58:10.102277+0000","last_clean":"2024-09-18T00:58:10.102277+0000","last_became_active":"2024-09-18T00:57:36.904791+0000","last_became_peered":"2024-09-18T00:57:36.904791+0000","last_unstale":"2024-09-18T00:58:10.102277+0000","last_undegraded":"2024-09-18T00:58:10.102277+0000","last_fullsized":"2024-09-18T00:58:10.102277+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:44.950474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090629000000000003,"stat_sum":{"num_bytes":1619307,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":621,"num_read_kb":761,"num_write":86,"num_write_kb":452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"48'173","reported_seq":513,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.124479+0000","last_change":"2024-09-18T00:57:38.909597+0000","last_active":"2024-09-18T00:58:10.124479+0000","last_peered":"2024-09-18T00:58:10.124479+0000","last_clean":"2024-09-18T00:58:10.124479+0000","last_became_active":"2024-09-18T00:57:36.905462+0000","last_became_peered":"2024-09-18T00:57:36.905462+0000","last_unstale":"2024-09-18T00:58:10.124479+0000","last_undegraded":"2024-09-18T00:58:10.124479+0000","last_fullsized":"2024-09-18T00:58:10.124479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:31:19.739280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081831799999999995,"stat_sum":{"num_bytes":12325442,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":805,"num_read_kb":6678,"num_write":222,"num_write_kb":4394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"49'167","reported_seq":1842,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.127417+0000","last_change":"2024-09-18T00:57:38.900906+0000","last_active":"2024-09-18T00:58:10.127417+0000","last_peered":"2024-09-18T00:58:10.127417+0000","last_clean":"2024-09-18T00:58:10.127417+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T00:58:10.127417+0000","last_undegraded":"2024-09-18T00:58:10.127417+0000","last_fullsized":"2024-09-18T00:58:10.127417+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:50:21.753082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069386899999999995,"stat_sum":{"num_bytes":31542742,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":26,"num_read":2258,"num_read_kb":33227,"num_write":142,"num_write_kb":23291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"48'82","reported_seq":229,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.099919+0000","last_change":"2024-09-18T00:57:38.899297+0000","last_active":"2024-09-18T00:58:10.099919+0000","last_peered":"2024-09-18T00:58:10.099919+0000","last_clean":"2024-09-18T00:58:10.099919+0000","last_became_active":"2024-09-18T00:57:36.897289+0000","last_became_peered":"2024-09-18T00:57:36.897289+0000","last_unstale":"2024-09-18T00:58:10.099919+0000","last_undegraded":"2024-09-18T00:58:10.099919+0000","last_fullsized":"2024-09-18T00:58:10.099919+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:17.287284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071487799999999995,"stat_sum":{"num_bytes":12846580,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":92,"num_read_kb":920,"num_write":64,"num_write_kb":823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"50'165","reported_seq":1241,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.188788+0000","last_change":"2024-09-18T00:57:39.056761+0000","last_active":"2024-09-18T00:58:10.188788+0000","last_peered":"2024-09-18T00:58:10.188788+0000","last_clean":"2024-09-18T00:58:10.188788+0000","last_became_active":"2024-09-18T00:57:36.910991+0000","last_became_peered":"2024-09-18T00:57:36.910991+0000","last_unstale":"2024-09-18T00:58:10.188788+0000","last_undegraded":"2024-09-18T00:58:10.188788+0000","last_fullsized":"2024-09-18T00:58:10.188788+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:43:10.231641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055104700000000004,"stat_sum":{"num_bytes":12963930,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":1656,"num_read_kb":9699,"num_write":177,"num_write_kb":8226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"50'269","reported_seq":754,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-18T00:58:10.151824+0000","last_change":"2024-09-18T00:57:38.899783+0000","last_active":"2024-09-18T00:58:10.151824+0000","last_peered":"2024-09-18T00:58:10.151824+0000","last_clean":"2024-09-18T00:58:10.151824+0000","last_became_active":"2024-09-18T00:57:36.904999+0000","last_became_peered":"2024-09-18T00:57:36.904999+0000","last_unstale":"2024-09-18T00:58:10.151824+0000","last_undegraded":"2024-09-18T00:58:10.151824+0000","last_fullsized":"2024-09-18T00:58:10.151824+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:42:03.206887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070399799999999995,"stat_sum":{"num_bytes":3670098,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":1624,"num_read_kb":5086,"num_write":371,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":410019879,"num_objects":993,"num_object_clones":187,"num_object_copies":1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":993,"num_whiteouts":486,"num_read":40337,"num_read_kb":264252,"num_write":6022,"num_write_kb":254388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258236416,"data_stored":270911244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5146,"ondisk_log_size":5146,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62684,"kb_used_data":27336,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94309156,"statfs":{"total":96636764160,"available":96572575744,"internally_reserved":0,"allocated":27992064,"data_stored":27793745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509453,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73896,"kb_used_data":38544,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94297944,"statfs":{"total":96636764160,"available":96561094656,"internally_reserved":0,"allocated":39469056,"data_stored":40595919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77396,"kb_used_data":42020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94294444,"statfs":{"total":96636764160,"available":96557510656,"internally_reserved":0,"allocated":43028480,"data_stored":44278251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67956,"kb_used_data":28448,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94303884,"statfs":{"total":96636764160,"available":96567177216,"internally_reserved":0,"allocated":29130752,"data_stored":31989788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57136,"kb_used_data":21784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94314704,"statfs":{"total":96636764160,"available":96578256896,"internally_reserved":0,"allocated":22306816,"data_stored":23667539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509454,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85308,"kb_used_data":45780,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94286532,"statfs":{"total":96636764160,"available":96549408768,"internally_reserved":0,"allocated":46878720,"data_stored":47907008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79236,"kb_used_data":39728,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94292604,"statfs":{"total":96636764160,"available":96555626496,"internally_reserved":0,"allocated":40681472,"data_stored":41983826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49436,"kb_used_data":14092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322404,"statfs":{"total":96636764160,"available":96586141696,"internally_reserved":0,"allocated":14430208,"data_stored":15794271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40116224,"data_stored":41741829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45719552,"data_stored":47074643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21741568,"data_stored":23425542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28561408,"data_stored":31747772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":42463232,"data_stored":44036254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":38309888,"data_stored":39763554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":27422720,"data_stored":27551729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13901824,"data_stored":15569921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:58:11.694 INFO:tasks.thrashosds.thrasher:Setting 3.d to [6, 7, 4, 1] 2024-09-18T00:58:11.694 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.d', '6', '7', '4', '1'] 2024-09-18T00:58:11.694 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.d 6 7 4 1 2024-09-18T00:58:11.695 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:11.682+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:11.789 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:11.786+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:11.861 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:11.861 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:11.862 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:11.862 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:11.878 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:11.891 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:11.886+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:1617 write 0xcb92ba2 thru 0xcb986a5 (0x5b04 bytes) 2024-09-18T00:58:11.948 INFO:teuthology.orchestra.run.smithi042.stdout:1620 trunc from 0xcb986a6 to 0x12f5278 2024-09-18T00:58:11.991 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:11.986+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:12.000 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:12.000 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:12.001 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:12.015 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:12.092 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:12.086+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:12.123 INFO:teuthology.orchestra.run.smithi042.stderr:set 3.d pg_upmap_items mapping to [6->7,4->1] 2024-09-18T00:58:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:58:12.185 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:12.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.106441544999999, 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.186 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:58:12.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.105511954000001, 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:58:12.190 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:58:12.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.105497893999999, 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:58:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:58:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:58:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:58:12.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.199 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.104207319, 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:58:12.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:58:12.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.104186847000001, 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.101392394000001, 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.101377209999999, 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.101354264000001, 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.097176624999999, 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.097155407999999, 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.097134024999999, 2024-09-18T00:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.093234170999999, 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.239 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.09321997, 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.093201933, 2024-09-18T00:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.093187867000001, 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.089162515999998, 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.089148504000001, 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.08913454, 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.327276586, 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.2018002150000004, 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:12.281 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout:1623 write 0x91579af thru 0x9160ba5 (0x91f7 bytes) 2024-09-18T00:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout:1627 read 0x7279843 thru 0x727dd46 (0x4504 bytes) 2024-09-18T00:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout:1628 read 0x8dbd5cb thru 0x8dc2aff (0x5535 bytes) 2024-09-18T00:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout:1629 read 0x7db771f thru 0x7dc1430 (0x9d12 bytes) 2024-09-18T00:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout:1631 read 0x4cb53e7 thru 0x4cbe1b9 (0x8dd3 bytes) 2024-09-18T00:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout:1633 read 0x22dd13c thru 0x22ec067 (0xef2c bytes) 2024-09-18T00:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout:1635 clone 17 order 19 su 16384 sc 11 2024-09-18T00:58:12.283 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:12.274+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:12.382 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:12.378+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:12.451 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:12.451 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:12.451 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:12.466 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:12.514 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:12.510+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:12.583 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:12.582+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:12.590 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:12.590 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:12.604 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:12.692 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:12.686+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.457225514999998, 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101264911, 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4226", 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2155402750", 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313760+0000", 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.612e-06 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313768+0000", 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9249999999999995e-06 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313786+0000", 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8253999999999998e-05 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313803+0000", 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313935+0000", 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013180399999999999 2024-09-18T00:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313954+0000", 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8768999999999999e-05 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.411874+0000", 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.097920646 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412292+0000", 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041777699999999998 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412675+0000", 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038262300000000001 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414019+0000", 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013436699999999999 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414872+0000", 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085320999999999997 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414955+0000", 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3405999999999994e-05 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.415023+0000", 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7825999999999997e-05 2024-09-18T00:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3739 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867528+0000", 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.903455371, 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020284267000000002, 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3739 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867528+0000", 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867528+0000", 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867529+0000", 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6400000000000002e-07 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867531+0000", 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867539+0000", 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1520000000000003e-06 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867555+0000", 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6566999999999999e-05 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887588+0000", 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020032395000000001 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887691+0000", 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103582 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887812+0000", 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121344 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3740 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.903420788999998, 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020503747999999999, 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3740 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867566+0000", 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3159999999999999e-06 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867571+0000", 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9240000000000004e-06 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867576+0000", 2024-09-18T00:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8210000000000001e-06 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887847+0000", 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020270468 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887952+0000", 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010541599999999999 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888067+0000", 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000114391 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3741 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.903400909999998, 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020744149999999999, 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3741 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0800000000000001e-07 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867585+0000", 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867591+0000", 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3269999999999997e-06 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867597+0000", 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0750000000000003e-06 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888108+0000", 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020510589999999999 2024-09-18T00:58:12.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888212+0000", 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104252 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888327+0000", 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011461800000000001 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3742 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.903379379, 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020983208999999999, 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3742 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.796 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867605+0000", 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867613+0000", 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867620+0000", 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.793e-06 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867625+0000", 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2619999999999998e-06 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888362+0000", 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020736747999999999 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888469+0000", 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106969 2024-09-18T00:58:12.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888587+0000", 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118904 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3743 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.903351785000002, 2024-09-18T00:58:12.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021178947, 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3743 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867634+0000", 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.956e-06 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867640+0000", 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0769999999999996e-06 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867645+0000", 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5619999999999997e-06 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888622+0000", 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020977554999999998 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888715+0000", 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2905000000000006e-05 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888811+0000", 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5513999999999994e-05 2024-09-18T00:58:12.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3744 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.903332506000002, 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021391389, 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3744 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8299999999999998e-07 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867653+0000", 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867659+0000", 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5999999999999997e-06 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867664+0000", 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8910000000000003e-06 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888846+0000", 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021181867 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888951+0000", 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105837 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889042+0000", 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1012000000000002e-05 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3745 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.903311252999998, 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021600477999999999, 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3745 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.810 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867673+0000", 2024-09-18T00:58:12.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2599999999999998e-07 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867675+0000", 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0499999999999999e-06 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867682+0000", 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0140000000000001e-06 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867687+0000", 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9790000000000001e-06 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889079+0000", 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021392347999999999 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889182+0000", 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102712 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889273+0000", 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1049000000000002e-05 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3746 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.903289665999999, 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021835075999999998, 2024-09-18T00:58:12.814 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3746 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6100000000000002e-07 2024-09-18T00:58:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867701+0000", 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867709+0000", 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7999999999999999e-06 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867715+0000", 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.699e-06 2024-09-18T00:58:12.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889307+0000", 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021592239999999999 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889410+0000", 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103093 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889529+0000", 2024-09-18T00:58:12.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011872299999999999 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3747 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.903259517999999, 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022084972000000001, 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3747 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867725+0000", 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5199999999999996e-07 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867735+0000", 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7790000000000007e-06 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867743+0000", 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1389999999999995e-06 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867748+0000", 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7209999999999999e-06 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889588+0000", 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021839447000000001 2024-09-18T00:58:12.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889706+0000", 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117748 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889809+0000", 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103486 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3748 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.903224339000001, 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022206404999999999, 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3748 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.824 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867760+0000", 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-07 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867762+0000", 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6079999999999998e-06 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867769+0000", 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9840000000000004e-06 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867785+0000", 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5616000000000001e-05 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889845+0000", 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022060283 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889913+0000", 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8104999999999999e-05 2024-09-18T00:58:12.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889966+0000", 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2340999999999999e-05 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7282 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473428+0000", 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.297555427999999, 2024-09-18T00:58:12.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020338440999999999, 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7282 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473428+0000", 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473428+0000", 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473429+0000", 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.23e-07 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473434+0000", 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7740000000000004e-06 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473449+0000", 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4922e-05 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473481+0000", 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2382999999999998e-05 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493569+0000", 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020087588 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493640+0000", 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1237999999999998e-05 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493767+0000", 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126713 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.832 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7283 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1703936~2106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.297483128, 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02059619, 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7283 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473501+0000", 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473507+0000", 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2939999999999996e-06 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473522+0000", 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5386000000000001e-05 2024-09-18T00:58:12.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473534+0000", 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1917e-05 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493796+0000", 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020262184999999999 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493907+0000", 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011105800000000001 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494097+0000", 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018939299999999999 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.837 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7284 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.297433188999999, 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020722913999999999, 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7284 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473551+0000", 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999999e-07 2024-09-18T00:58:12.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473556+0000", 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8949999999999997e-06 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473571+0000", 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.486e-05 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473603+0000", 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2375000000000001e-05 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494129+0000", 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020525244000000002 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494206+0000", 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7601000000000005e-05 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494273+0000", 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7101999999999996e-05 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7292 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.297362918000001, 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020882831000000001, 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7292 2024-09-18T00:58:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:12.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473622+0000", 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.78e-07 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473626+0000", 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7330000000000003e-06 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473640+0000", 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4034e-05 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473651+0000", 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0937999999999999e-05 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494300+0000", 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020649238 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494392+0000", 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1184e-05 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494504+0000", 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111826 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7293 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.297317601, 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021019589000000002, 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7293 2024-09-18T00:58:12.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473667+0000", 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6600000000000005e-07 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473682+0000", 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.483e-05 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.477101+0000", 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034192710000000002 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.477120+0000", 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9202999999999999e-05 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494533+0000", 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017412838999999999 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494603+0000", 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9947e-05 2024-09-18T00:58:12.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494686+0000", 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2633000000000007e-05 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15500 3.11 3:8ed6a97b:::rbd_data.117c53e529c1.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.956465495, 2024-09-18T00:58:12.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046818956000000002, 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15500 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814521+0000", 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.977e-06 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814536+0000", 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.487e-05 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814572+0000", 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5567e-05 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814588+0000", 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6546e-05 2024-09-18T00:58:12.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814675+0000", 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6675999999999995e-05 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814839+0000", 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016414199999999999 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:12.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.819908+0000", 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050695280000000002 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861139+0000", 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041230771999999999 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861208+0000", 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8722999999999999e-05 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861277+0000", 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9246999999999996e-05 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861337+0000", 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9908000000000003e-05 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.857 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:21989 3.1d 3:beac871e:::rbd_object_map.117c21a2e20a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:58:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.241598859, 2024-09-18T00:58:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021103017000000002, 2024-09-18T00:58:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.858 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 21989 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2399999999999999e-07 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529389+0000", 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3759999999999999e-06 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529398+0000", 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4080000000000004e-06 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529404+0000", 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8919999999999997e-06 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.544686+0000", 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015282662000000001 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.544799+0000", 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011215800000000001 2024-09-18T00:58:12.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.545254+0000", 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045577600000000002 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.547172+0000", 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019178999999999999 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550317+0000", 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031449310000000001 2024-09-18T00:58:12.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550437+0000", 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119963 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550488+0000", 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0526999999999999e-05 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:21990 3.1d 3:beac871e:::rbd_object_map.117c21a2e20a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.241570412, 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023815820000000001, 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 21990 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529414+0000", 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529417+0000", 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2849999999999999e-06 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529425+0000", 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7740000000000004e-06 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529431+0000", 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8939999999999998e-06 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546090+0000", 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016659706 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546185+0000", 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4889999999999994e-05 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546569+0000", 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038352599999999997 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.547692+0000", 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001123024 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553127+0000", 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054353630000000003 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553188+0000", 2024-09-18T00:58:12.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1247999999999994e-05 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553229+0000", 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0646999999999999e-05 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26860 3.a 3:523ff797:::rbd_data.117c4c7db1a0.0000000000000039:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:12.869 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3982913759999995, 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028475341000000001, 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26860 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372693+0000", 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0550000000000001e-06 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372699+0000", 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372714+0000", 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4673e-05 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372722+0000", 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4659999999999999e-06 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372787+0000", 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5288000000000002e-05 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372874+0000", 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6485999999999998e-05 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.377992+0000", 2024-09-18T00:58:12.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051177159999999996 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401008+0000", 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02301603 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401026+0000", 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712e-05 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:12.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401082+0000", 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5648e-05 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401168+0000", 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5476999999999995e-05 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:12.875 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:12.877 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:12.877 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:12.870+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:12.978 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:12.974+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:13.008 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:13.008 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:13.022 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:13.054 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:13.109 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:13.106+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:13.180 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:13.174+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:13.186 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:13.186 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:13.186 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:13.186 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:13.186 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:13.201 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:13.219 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:13.220 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:13.239 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:13.240 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:13.251 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone16 from 0x9160ba6 (overlap 0x12f5278) to 0x0 2024-09-18T00:58:13.261 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:13.261 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:13.289 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:13.286+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:13.325 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:13.345 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:13.367 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:13.367 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:13.382 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:13.378+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:13.383 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:13.383 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:13.400 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:13.483 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:13.478+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:13.583 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:13.584 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:13.584 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:13.578+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:13.603 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:13.689 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:13.686+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:13.793 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:13.782+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:13.814 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:13.814 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:13.834 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:13.921 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:13.918+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:13.988 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:13.982+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:14.030 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:14.030 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:14.048 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:14.089 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:14.082+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:14.150 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:58:14.194 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:14.190+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:14.246 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:14.246 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:14.263 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:14.309 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:14.306+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:14.392 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:14.386+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:14.426 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:14.426 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:14.444 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:14.493 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:14.486+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:14.594 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:14.590+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:14.600 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:14.600 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:14.613 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:14.695 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:14.690+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:14.731 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:14.731 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:14.732 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-18T00:58:14.745 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:14.796 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:14.790+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:14.897 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:14.894+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:14.907 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:14.907 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:14.920 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:15.009 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:15.006+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:15.077 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:15.077 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:15.091 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:15.127 INFO:teuthology.orchestra.run.smithi042.stdout:1636 read 0x37520ed thru 0x3756ae1 (0x49f5 bytes) 2024-09-18T00:58:15.133 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:15.130+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:15.136 INFO:teuthology.orchestra.run.smithi042.stdout:1637 trunc from 0x9160ba6 to 0xbe0f77f 2024-09-18T00:58:15.145 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:58:15.157 INFO:teuthology.orchestra.run.smithi042.stdout:1638 trunc from 0xbe0f77f to 0x6142feb 2024-09-18T00:58:15.182 INFO:teuthology.orchestra.run.smithi042.stdout:1640 read 0x2ef92d8 thru 0x2efe2c4 (0x4fed bytes) 2024-09-18T00:58:15.198 INFO:teuthology.orchestra.run.smithi042.stdout:1641 writesame 0xb70a0a3 thru 0xb71202b data_size 0x1(0x7f89 bytes) 2024-09-18T00:58:15.199 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:15.194+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:15.206 INFO:teuthology.orchestra.run.smithi042.stdout:1642 write 0xc7a449e thru 0xc7a75b0 (0x3113 bytes) 2024-09-18T00:58:15.211 INFO:teuthology.orchestra.run.smithi042.stdout:1643 write 0x15078f4 thru 0x150b2aa (0x39b7 bytes) 2024-09-18T00:58:15.242 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:15.242 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:15.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:15.264 INFO:teuthology.orchestra.run.smithi042.stdout:1644 read 0x1815868 thru 0x181d384 (0x7b1d bytes) 2024-09-18T00:58:15.279 INFO:teuthology.orchestra.run.smithi042.stdout:1645 trunc from 0xc7a75b1 to 0x38148d3 2024-09-18T00:58:15.299 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:15.294+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:15.330 INFO:teuthology.orchestra.run.smithi042.stdout:1646 writesame 0xe525596 thru 0xe52c566 data_size 0x5(0x6fd1 bytes) 2024-09-18T00:58:15.330 INFO:teuthology.orchestra.run.smithi042.stdout:1647 read 0xb6e23c7 thru 0xb6e74e3 (0x511d bytes) 2024-09-18T00:58:15.330 INFO:teuthology.orchestra.run.smithi042.stdout:1648 punch from 0xafe9acd to 0xafea5dc, (0xb0f bytes) 2024-09-18T00:58:15.331 INFO:teuthology.orchestra.run.smithi042.stdout:1650 read 0x25d69dc thru 0x25dfd1d (0x9342 bytes) 2024-09-18T00:58:15.331 INFO:teuthology.orchestra.run.smithi042.stdout:1651 write 0xc2cb289 thru 0xc2db110 (0xfe88 bytes) 2024-09-18T00:58:15.343 INFO:teuthology.orchestra.run.smithi042.stdout:1653 read 0x3d2a76f thru 0x3d34eb4 (0xa746 bytes) 2024-09-18T00:58:15.343 INFO:teuthology.orchestra.run.smithi042.stdout:1654 write 0xa4a347c thru 0xa4a4e23 (0x19a8 bytes) 2024-09-18T00:58:15.357 INFO:teuthology.orchestra.run.smithi042.stdout:1656 trunc from 0xe52c567 to 0x83d0807 2024-09-18T00:58:15.368 INFO:teuthology.orchestra.run.smithi042.stdout:1657 read 0x643cbce thru 0x644bff4 (0xf427 bytes) 2024-09-18T00:58:15.369 INFO:teuthology.orchestra.run.smithi042.stdout:1659 read 0x39a7161 thru 0x39b6ea6 (0xfd46 bytes) 2024-09-18T00:58:15.376 INFO:teuthology.orchestra.run.smithi042.stdout:1660 trunc from 0x83d0807 to 0xc7c3d52 2024-09-18T00:58:15.381 INFO:teuthology.orchestra.run.smithi042.stdout:1662 clone 18 order 21 su 1048576 sc 10 2024-09-18T00:58:15.387 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:15.411 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:15.412 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:15.412 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:15.406+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:15.427 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:15.502 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:15.498+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:15.536 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:15.536 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:15.554 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:15.554 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:15.571 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:15.571 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:15.588 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:15.588 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:15.603 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:15.598+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.540970492, 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:15.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.608 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:58:15.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.608 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:15.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:15.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:58:15.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:58:15.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:15.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:58:15.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:58:15.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:15.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:58:15.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:58:15.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:15.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:58:15.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:58:15.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:15.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:58:15.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:58:15.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28571 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16109568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.538271+0000", 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0666641050000001, 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049622299000000002, 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28571 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538271+0000", 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538271+0000", 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538272+0000", 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3599999999999999e-07 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538278+0000", 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538285+0000", 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.46e-06 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538305+0000", 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0545000000000001e-05 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.587723+0000", 2024-09-18T00:58:15.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049417852999999998 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.587808+0000", 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5063999999999997e-05 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.587894+0000", 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5735000000000005e-05 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28572 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16121856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.538314+0000", 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0666212719999999, 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049780243000000002, 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28572 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.619 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538314+0000", 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538314+0000", 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.541764+0000", 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034495239999999998 2024-09-18T00:58:15.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.541769+0000", 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3539999999999999e-06 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.541784+0000", 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5221999999999999e-05 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.541793+0000", 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4300000000000006e-06 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.587936+0000", 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046143211000000003 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.588015+0000", 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8961999999999994e-05 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.588094+0000", 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9540000000000001e-05 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28590 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16343040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542264+0000", 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0626715860000004, 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049631623, 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28590 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542264+0000", 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542264+0000", 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542264+0000", 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.75e-07 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542267+0000", 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3879999999999998e-06 2024-09-18T00:58:15.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542274+0000", 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.844e-06 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542279+0000", 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1340000000000003e-06 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.591736+0000", 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049457713 2024-09-18T00:58:15.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.591816+0000", 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9722999999999995e-05 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.591896+0000", 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9345999999999996e-05 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.627 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28591 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16355328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542287+0000", 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0626490090000003, 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049828273999999999, 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28591 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542287+0000", 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542287+0000", 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542287+0000", 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542289+0000", 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.424e-06 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542306+0000", 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6639000000000001e-05 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542312+0000", 2024-09-18T00:58:15.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6659999999999996e-06 2024-09-18T00:58:15.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.591941+0000", 2024-09-18T00:58:15.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049629672999999999 2024-09-18T00:58:15.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592028+0000", 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6810000000000004e-05 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592115+0000", 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6621999999999998e-05 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.632 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28592 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16367616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542320+0000", 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0626153079999998, 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049995825000000001, 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28592 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542320+0000", 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542320+0000", 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542321+0000", 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-07 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542323+0000", 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6759999999999999e-06 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542330+0000", 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9190000000000002e-06 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542336+0000", 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3589999999999999e-06 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592157+0000", 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049821097000000002 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592234+0000", 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7729000000000001e-05 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592316+0000", 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1559999999999998e-05 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28593 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16379904~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542361+0000", 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0625741380000004, 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050172784999999998, 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28593 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542361+0000", 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542361+0000", 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542362+0000", 2024-09-18T00:58:15.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0399999999999996e-07 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542365+0000", 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.864e-06 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542373+0000", 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0979999999999994e-06 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542379+0000", 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7270000000000004e-06 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592362+0000", 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049982930000000002 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592446+0000", 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4442000000000002e-05 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592534+0000", 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8120000000000003e-05 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28594 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16392192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542387+0000", 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.062548864, 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050350209999999999, 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28594 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542387+0000", 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542387+0000", 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542387+0000", 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3200000000000005e-07 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542389+0000", 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0980000000000001e-06 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542403+0000", 2024-09-18T00:58:15.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4205e-05 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542411+0000", 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4710000000000001e-06 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592577+0000", 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050165609 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592658+0000", 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1848000000000006e-05 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592737+0000", 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8447000000000003e-05 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28595 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16404480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542419+0000", 2024-09-18T00:58:15.645 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0625167959999997, 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050463569, 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28595 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542419+0000", 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542419+0000", 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542419+0000", 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3599999999999999e-07 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542426+0000", 2024-09-18T00:58:15.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9210000000000003e-06 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542433+0000", 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8959999999999997e-06 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542448+0000", 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4701999999999999e-05 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592780+0000", 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050331819 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592831+0000", 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1533999999999998e-05 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592882+0000", 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1261000000000003e-05 2024-09-18T00:58:15.649 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28596 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16416768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542455+0000", 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0624807240000003, 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050626526999999998, 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28596 2024-09-18T00:58:15.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.651 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542455+0000", 2024-09-18T00:58:15.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542455+0000", 2024-09-18T00:58:15.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542455+0000", 2024-09-18T00:58:15.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7100000000000001e-07 2024-09-18T00:58:15.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542458+0000", 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1770000000000001e-06 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542464+0000", 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2709999999999998e-06 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542469+0000", 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7500000000000003e-06 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592926+0000", 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050457098999999998 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593004+0000", 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8147999999999998e-05 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593081+0000", 2024-09-18T00:58:15.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7410999999999995e-05 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28597 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16429056~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542485+0000", 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0624508119999998, 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050727758999999997, 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28597 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542485+0000", 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542485+0000", 2024-09-18T00:58:15.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542485+0000", 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-07 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542493+0000", 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542510+0000", 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6135999999999999e-05 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542519+0000", 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.183e-06 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593119+0000", 2024-09-18T00:58:15.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050600539 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593166+0000", 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6220999999999997e-05 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593212+0000", 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6896e-05 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28598 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16441344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542529+0000", 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0624065800000002, 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050882327999999998, 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28598 2024-09-18T00:58:15.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542529+0000", 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542529+0000", 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.545724+0000", 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031947960000000002 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.545729+0000", 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7579999999999998e-06 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.545742+0000", 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2877999999999999e-05 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.545750+0000", 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0080000000000002e-06 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593253+0000", 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.047502731999999999 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593334+0000", 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1162999999999995e-05 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593411+0000", 2024-09-18T00:58:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6993e-05 2024-09-18T00:58:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28614 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16637952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546234+0000", 2024-09-18T00:58:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0587018720000003, 2024-09-18T00:58:15.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049626101999999998, 2024-09-18T00:58:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28614 2024-09-18T00:58:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546234+0000", 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546234+0000", 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546234+0000", 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546237+0000", 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5129999999999999e-06 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546252+0000", 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5727999999999999e-05 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546259+0000", 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2659999999999998e-06 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595779+0000", 2024-09-18T00:58:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049520284999999997 2024-09-18T00:58:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595818+0000", 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9542999999999997e-05 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595860+0000", 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1337000000000001e-05 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.669 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28615 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16650240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546267+0000", 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0586687469999996, 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049708853999999997, 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28615 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546267+0000", 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546267+0000", 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546267+0000", 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9299999999999998e-07 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546270+0000", 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7439999999999999e-06 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546278+0000", 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6620000000000005e-06 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546283+0000", 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1290000000000004e-06 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595891+0000", 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049608273000000001 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595929+0000", 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7877999999999999e-05 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595976+0000", 2024-09-18T00:58:15.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6675000000000003e-05 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28616 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16662528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546291+0000", 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0586450220000003, 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049808481000000002, 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28616 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546291+0000", 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546291+0000", 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546291+0000", 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8800000000000003e-07 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546293+0000", 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3429999999999998e-06 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546308+0000", 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4833e-05 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546314+0000", 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0689999999999998e-06 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596018+0000", 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049703658999999997 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596058+0000", 2024-09-18T00:58:15.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0170999999999999e-05 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596099+0000", 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0917999999999997e-05 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28617 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16674816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546346+0000", 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0585891930000004, 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049862966000000002, 2024-09-18T00:58:15.678 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28617 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546346+0000", 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546346+0000", 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546347+0000", 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5899999999999996e-07 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546350+0000", 2024-09-18T00:58:15.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9919999999999999e-06 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546366+0000", 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5698000000000001e-05 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546372+0000", 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5649999999999998e-06 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596128+0000", 2024-09-18T00:58:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049756250000000002 2024-09-18T00:58:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596166+0000", 2024-09-18T00:58:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7586999999999998e-05 2024-09-18T00:58:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596209+0000", 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3315000000000001e-05 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28618 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16687104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546382+0000", 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0585536590000002, 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049948445000000001, 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28618 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546382+0000", 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546382+0000", 2024-09-18T00:58:15.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546382+0000", 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546385+0000", 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6570000000000001e-06 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546394+0000", 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1220000000000001e-06 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546400+0000", 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6640000000000003e-06 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596250+0000", 2024-09-18T00:58:15.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049849656999999999 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596289+0000", 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9681000000000002e-05 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596330+0000", 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1050000000000002e-05 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28619 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16699392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546412+0000", 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0585238820000002, 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050029523999999999, 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28619 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546412+0000", 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546412+0000", 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546412+0000", 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-07 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546421+0000", 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5110000000000003e-06 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546428+0000", 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1930000000000004e-06 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546433+0000", 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2059999999999998e-06 2024-09-18T00:58:15.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596360+0000", 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049926816999999998 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596398+0000", 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7712000000000001e-05 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596441+0000", 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3616999999999999e-05 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28620 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16711680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546465+0000", 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0584703969999998, 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050094769999999997, 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28620 2024-09-18T00:58:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546465+0000", 2024-09-18T00:58:15.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546465+0000", 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546466+0000", 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0999999999999998e-07 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546469+0000", 2024-09-18T00:58:15.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8720000000000002e-06 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546477+0000", 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0160000000000008e-06 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546483+0000", 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9529999999999996e-06 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596482+0000", 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049998919000000003 2024-09-18T00:58:15.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596520+0000", 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8359000000000003e-05 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596560+0000", 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9941000000000003e-05 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28621 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16723968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546492+0000", 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0584435550000002, 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050179060999999997, 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28621 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.697 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546492+0000", 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546492+0000", 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.549801+0000", 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033093630000000001 2024-09-18T00:58:15.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.549807+0000", 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8170000000000004e-06 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.549818+0000", 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1139e-05 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.549828+0000", 2024-09-18T00:58:15.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9389999999999992e-06 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596590+0000", 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046761745 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596628+0000", 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7601999999999997e-05 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596671+0000", 2024-09-18T00:58:15.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3455999999999999e-05 2024-09-18T00:58:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:15.701 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:15.701 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:15.701 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:15.702 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:15.703 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:15.703 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:15.703 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:15.704 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:15.704 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:15.704 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:15.705 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:15.706 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:15.702+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:15.808 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:15.806+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:15.878 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:15.878 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:15.878 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:15.878 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:15.892 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:15.908 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:15.906+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:16.009 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:16.002+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:16.027 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:16.027 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:16.027 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:16.027 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:16.027 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:16.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:16.130 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:16.126+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:16.212 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:16.210+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:16.222 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:58:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.930745438999999, 2024-09-18T00:58:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:58:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:16.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:58:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:58:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:58:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:58:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:58:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:58:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:58:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:58:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:58:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:58:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:58:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:58:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:58:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:58:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:58:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:58:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:58:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:58:16.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:58:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:58:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:58:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:58:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:58:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:16.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:58:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:58:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:58:16.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:58:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:58:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:58:16.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7423 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.237 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.734446889000001, 2024-09-18T00:58:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045296661000000002, 2024-09-18T00:58:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7423 2024-09-18T00:58:16.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.239 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:16.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:16.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:16.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-18T00:58:16.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485621+0000", 2024-09-18T00:58:16.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:58:16.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485627+0000", 2024-09-18T00:58:16.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.156e-06 2024-09-18T00:58:16.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485633+0000", 2024-09-18T00:58:16.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789999999999999e-06 2024-09-18T00:58:16.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530788+0000", 2024-09-18T00:58:16.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045155001 2024-09-18T00:58:16.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530853+0000", 2024-09-18T00:58:16.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5634000000000002e-05 2024-09-18T00:58:16.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530915+0000", 2024-09-18T00:58:16.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2022999999999999e-05 2024-09-18T00:58:16.246 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.246 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.247 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7424 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1146880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.247 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:16.247 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.734425899000001, 2024-09-18T00:58:16.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045419355000000002, 2024-09-18T00:58:16.248 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7424 2024-09-18T00:58:16.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.249 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:16.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:16.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:16.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:16.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485642+0000", 2024-09-18T00:58:16.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0640000000000001e-06 2024-09-18T00:58:16.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485649+0000", 2024-09-18T00:58:16.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-18T00:58:16.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485654+0000", 2024-09-18T00:58:16.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2329999999999997e-06 2024-09-18T00:58:16.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530941+0000", 2024-09-18T00:58:16.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045286602000000002 2024-09-18T00:58:16.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531004+0000", 2024-09-18T00:58:16.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3455999999999997e-05 2024-09-18T00:58:16.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531059+0000", 2024-09-18T00:58:16.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4771000000000001e-05 2024-09-18T00:58:16.256 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.256 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.256 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.257 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.257 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:16.257 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.734404386000001, 2024-09-18T00:58:16.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:58:16.257 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.257 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.257 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:58:16.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.258 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:16.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:16.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:16.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:16.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:58:16.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:58:16.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:58:16.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:58:16.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:58:16.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:58:16.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:58:16.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:58:16.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:58:16.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:58:16.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:58:16.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:58:16.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.266 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.266 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.267 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:16.267 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.734382949, 2024-09-18T00:58:16.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:58:16.267 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:58:16.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.268 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:16.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:16.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:58:16.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:58:16.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:58:16.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:58:16.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:58:16.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:58:16.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:58:16.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:58:16.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:58:16.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:58:16.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:58:16.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:58:16.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:58:16.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:58:16.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.734287622, 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:58:16.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:58:16.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.734262182999998, 2024-09-18T00:58:16.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:16.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:58:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.284 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.285 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.285 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:16.285 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.73423966, 2024-09-18T00:58:16.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:58:16.285 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:58:16.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.286 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:58:16.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:58:16.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:58:16.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.290 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.73421622, 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:16.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:58:16.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.734192409999999, 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.295 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:16.299 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.734170044999999, 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:16.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.303 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.734143427999999, 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.305 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:58:16.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.305 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:16.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:58:16.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:58:16.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.308 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.729991656999999, 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.309 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:58:16.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.313 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.729970913999999, 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:16.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:16.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:58:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:58:16.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:58:16.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:58:16.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:58:16.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.318 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.729925348999998, 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:58:16.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:58:16.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:58:16.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:58:16.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.323 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.323 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.323 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.729899008, 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:58:16.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:58:16.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.729876338, 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.328 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.332 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.729857965000001, 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:58:16.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.729838177000001, 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.337 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.338 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:58:16.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.338 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:16.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:58:16.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:58:16.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:58:16.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:58:16.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:58:16.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.729811306999999, 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:58:16.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.346 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:16.348 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:16.348 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:16.342+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:16.366 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone17 2024-09-18T00:58:16.448 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:16.442+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:16.542 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:16.543 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:16.543 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:16.549 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:16.542+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:16.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:16.650 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:16.646+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:16.745 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:16.745 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:16.745 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:16.745 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:16.745 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:16.751 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:16.746+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:16.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:16.851 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:16.846+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:16.952 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:16.946+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:16.960 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:16.960 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.881456710999998, 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.880527119999996, 2024-09-18T00:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.880513059999998, 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.879222485, 2024-09-18T00:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.879202012999997, 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.876407559999997, 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.876392375999998, 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.876369429999997, 2024-09-18T00:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.872191790999999, 2024-09-18T00:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.872170574000002, 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.872149190999998, 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.868249337000002, 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.868235136000003, 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.868217098999999, 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.868203033, 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.864177681999998, 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.864163670000003, 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.864149705999999, 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.102291751999999, 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.9768153809999998, 2024-09-18T00:58:17.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:58:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:58:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:58:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.056 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.056 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:17.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:17.058 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:17.054+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:17.149 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:58:17.150 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T00:58:17.179 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:58:17.189 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:17.189 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:17.189 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:17.189 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:17.190 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:17.182+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:17.209 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:17.259 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:17.254+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:17.314 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-18T00:58:17.327 INFO:teuthology.orchestra.run.smithi042.stdout:1663 write 0x3184bbf thru 0x318fe26 (0xb268 bytes) 2024-09-18T00:58:17.340 INFO:teuthology.orchestra.run.smithi042.stdout:1664 read 0x828ed2d thru 0x829bf53 (0xd227 bytes) 2024-09-18T00:58:17.352 INFO:teuthology.orchestra.run.smithi042.stdout:1665 writesame 0xdf2c5ad thru 0xdf39a15 data_size 0x1(0xd469 bytes) 2024-09-18T00:58:17.352 INFO:teuthology.orchestra.run.smithi042.stdout:1670 read 0xcee327a thru 0xcee4447 (0x11ce bytes) 2024-09-18T00:58:17.352 INFO:teuthology.orchestra.run.smithi042.stdout:1671 punch from 0x1874329 to 0x187511b, (0xdf2 bytes) 2024-09-18T00:58:17.353 INFO:teuthology.orchestra.run.smithi042.stdout:1672 write 0xa048b4e thru 0xa04a462 (0x1915 bytes) 2024-09-18T00:58:17.356 INFO:teuthology.orchestra.run.smithi042.stdout:1674 punch from 0x925b8c0 to 0x925c14b, (0x88b bytes) 2024-09-18T00:58:17.356 INFO:teuthology.orchestra.run.smithi042.stdout:1676 write 0xc2c0f9 thru 0xc37010 (0xaf18 bytes) 2024-09-18T00:58:17.360 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:17.354+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:17.362 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:17.362 INFO:teuthology.orchestra.run.smithi042.stdout:1679 read 0x7c02419 thru 0x7c090f2 (0x6cda bytes) 2024-09-18T00:58:17.362 INFO:teuthology.orchestra.run.smithi042.stdout:1681 write 0x60c9850 thru 0x60d2554 (0x8d05 bytes) 2024-09-18T00:58:17.369 INFO:teuthology.orchestra.run.smithi042.stdout:1682 clone 19 order 23 su 32768 sc 9 2024-09-18T00:58:17.398 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:17.399 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:17.413 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:17.461 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:17.458+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:17.531 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:17.531 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:17.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:17.531 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:17.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.531 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:58:17.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.214317971, 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101264911, 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4226", 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2155402750", 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:17.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313760+0000", 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.612e-06 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313768+0000", 2024-09-18T00:58:17.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9249999999999995e-06 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313786+0000", 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8253999999999998e-05 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313803+0000", 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313935+0000", 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013180399999999999 2024-09-18T00:58:17.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313954+0000", 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8768999999999999e-05 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.411874+0000", 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.097920646 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412292+0000", 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041777699999999998 2024-09-18T00:58:17.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412675+0000", 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038262300000000001 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414019+0000", 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013436699999999999 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414872+0000", 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085320999999999997 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414955+0000", 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3405999999999994e-05 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.415023+0000", 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7825999999999997e-05 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3739 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.537 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867528+0000", 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.660547827000002, 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020284267000000002, 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3739 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867528+0000", 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867528+0000", 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867529+0000", 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6400000000000002e-07 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867531+0000", 2024-09-18T00:58:17.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867539+0000", 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1520000000000003e-06 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867555+0000", 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6566999999999999e-05 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887588+0000", 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020032395000000001 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887691+0000", 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103582 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887812+0000", 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121344 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3740 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.660513244999997, 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020503747999999999, 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3740 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.542 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867566+0000", 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3159999999999999e-06 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867571+0000", 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9240000000000004e-06 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867576+0000", 2024-09-18T00:58:17.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8210000000000001e-06 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887847+0000", 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020270468 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887952+0000", 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010541599999999999 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888067+0000", 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000114391 2024-09-18T00:58:17.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3741 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.660493365999997, 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020744149999999999, 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3741 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0800000000000001e-07 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867585+0000", 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867591+0000", 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3269999999999997e-06 2024-09-18T00:58:17.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867597+0000", 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0750000000000003e-06 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888108+0000", 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020510589999999999 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888212+0000", 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104252 2024-09-18T00:58:17.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888327+0000", 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011461800000000001 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3742 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.660471835000003, 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020983208999999999, 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3742 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867605+0000", 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867613+0000", 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867620+0000", 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.793e-06 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867625+0000", 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2619999999999998e-06 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888362+0000", 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020736747999999999 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888469+0000", 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106969 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888587+0000", 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118904 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3743 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.660444241, 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021178947, 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3743 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867634+0000", 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.956e-06 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867640+0000", 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0769999999999996e-06 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867645+0000", 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5619999999999997e-06 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888622+0000", 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020977554999999998 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888715+0000", 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2905000000000006e-05 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888811+0000", 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5513999999999994e-05 2024-09-18T00:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3744 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.660424962, 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021391389, 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3744 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8299999999999998e-07 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867653+0000", 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867659+0000", 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5999999999999997e-06 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867664+0000", 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8910000000000003e-06 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888846+0000", 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021181867 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888951+0000", 2024-09-18T00:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105837 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889042+0000", 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1012000000000002e-05 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3745 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.660403709000001, 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021600477999999999, 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3745 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867673+0000", 2024-09-18T00:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2599999999999998e-07 2024-09-18T00:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867675+0000", 2024-09-18T00:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0499999999999999e-06 2024-09-18T00:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867682+0000", 2024-09-18T00:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0140000000000001e-06 2024-09-18T00:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867687+0000", 2024-09-18T00:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9790000000000001e-06 2024-09-18T00:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889079+0000", 2024-09-18T00:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021392347999999999 2024-09-18T00:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889182+0000", 2024-09-18T00:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102712 2024-09-18T00:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889273+0000", 2024-09-18T00:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1049000000000002e-05 2024-09-18T00:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3746 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.660382122000001, 2024-09-18T00:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021835075999999998, 2024-09-18T00:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3746 2024-09-18T00:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6100000000000002e-07 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867701+0000", 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867709+0000", 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7999999999999999e-06 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867715+0000", 2024-09-18T00:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.699e-06 2024-09-18T00:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889307+0000", 2024-09-18T00:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021592239999999999 2024-09-18T00:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889410+0000", 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103093 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889529+0000", 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011872299999999999 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3747 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.660351974000001, 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022084972000000001, 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3747 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867725+0000", 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5199999999999996e-07 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867735+0000", 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7790000000000007e-06 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867743+0000", 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1389999999999995e-06 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867748+0000", 2024-09-18T00:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7209999999999999e-06 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889588+0000", 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021839447000000001 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889706+0000", 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117748 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889809+0000", 2024-09-18T00:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103486 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3748 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.660316795, 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022206404999999999, 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3748 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867760+0000", 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-07 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867762+0000", 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6079999999999998e-06 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867769+0000", 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9840000000000004e-06 2024-09-18T00:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867785+0000", 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5616000000000001e-05 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889845+0000", 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022060283 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889913+0000", 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8104999999999999e-05 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889966+0000", 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2340999999999999e-05 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7282 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473428+0000", 2024-09-18T00:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.054647884000001, 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020338440999999999, 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7282 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473428+0000", 2024-09-18T00:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473428+0000", 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473429+0000", 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.23e-07 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473434+0000", 2024-09-18T00:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7740000000000004e-06 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473449+0000", 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4922e-05 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473481+0000", 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2382999999999998e-05 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493569+0000", 2024-09-18T00:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020087588 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493640+0000", 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1237999999999998e-05 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493767+0000", 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126713 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7283 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1703936~2106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.054575583999998, 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02059619, 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7283 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473501+0000", 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473507+0000", 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2939999999999996e-06 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473522+0000", 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5386000000000001e-05 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473534+0000", 2024-09-18T00:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1917e-05 2024-09-18T00:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493796+0000", 2024-09-18T00:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020262184999999999 2024-09-18T00:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493907+0000", 2024-09-18T00:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011105800000000001 2024-09-18T00:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494097+0000", 2024-09-18T00:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018939299999999999 2024-09-18T00:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7284 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.054525644999998, 2024-09-18T00:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020722913999999999, 2024-09-18T00:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7284 2024-09-18T00:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473551+0000", 2024-09-18T00:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999999e-07 2024-09-18T00:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473556+0000", 2024-09-18T00:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8949999999999997e-06 2024-09-18T00:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473571+0000", 2024-09-18T00:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.486e-05 2024-09-18T00:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473603+0000", 2024-09-18T00:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2375000000000001e-05 2024-09-18T00:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494129+0000", 2024-09-18T00:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020525244000000002 2024-09-18T00:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494206+0000", 2024-09-18T00:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7601000000000005e-05 2024-09-18T00:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494273+0000", 2024-09-18T00:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7101999999999996e-05 2024-09-18T00:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7292 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.054455374, 2024-09-18T00:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020882831000000001, 2024-09-18T00:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7292 2024-09-18T00:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473622+0000", 2024-09-18T00:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.78e-07 2024-09-18T00:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473626+0000", 2024-09-18T00:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7330000000000003e-06 2024-09-18T00:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473640+0000", 2024-09-18T00:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4034e-05 2024-09-18T00:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473651+0000", 2024-09-18T00:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0937999999999999e-05 2024-09-18T00:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494300+0000", 2024-09-18T00:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020649238 2024-09-18T00:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494392+0000", 2024-09-18T00:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1184e-05 2024-09-18T00:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494504+0000", 2024-09-18T00:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111826 2024-09-18T00:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7293 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.054410056999998, 2024-09-18T00:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021019589000000002, 2024-09-18T00:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7293 2024-09-18T00:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473667+0000", 2024-09-18T00:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6600000000000005e-07 2024-09-18T00:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473682+0000", 2024-09-18T00:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.483e-05 2024-09-18T00:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.477101+0000", 2024-09-18T00:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034192710000000002 2024-09-18T00:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.477120+0000", 2024-09-18T00:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9202999999999999e-05 2024-09-18T00:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494533+0000", 2024-09-18T00:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017412838999999999 2024-09-18T00:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494603+0000", 2024-09-18T00:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9947e-05 2024-09-18T00:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494686+0000", 2024-09-18T00:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2633000000000007e-05 2024-09-18T00:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15500 3.11 3:8ed6a97b:::rbd_data.117c53e529c1.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.713557950999999, 2024-09-18T00:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046818956000000002, 2024-09-18T00:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15500 2024-09-18T00:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814521+0000", 2024-09-18T00:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.977e-06 2024-09-18T00:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814536+0000", 2024-09-18T00:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.487e-05 2024-09-18T00:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814572+0000", 2024-09-18T00:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5567e-05 2024-09-18T00:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814588+0000", 2024-09-18T00:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6546e-05 2024-09-18T00:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814675+0000", 2024-09-18T00:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6675999999999995e-05 2024-09-18T00:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814839+0000", 2024-09-18T00:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016414199999999999 2024-09-18T00:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.819908+0000", 2024-09-18T00:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050695280000000002 2024-09-18T00:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861139+0000", 2024-09-18T00:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041230771999999999 2024-09-18T00:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861208+0000", 2024-09-18T00:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8722999999999999e-05 2024-09-18T00:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861277+0000", 2024-09-18T00:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9246999999999996e-05 2024-09-18T00:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861337+0000", 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9908000000000003e-05 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:21989 3.1d 3:beac871e:::rbd_object_map.117c21a2e20a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.998691315, 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021103017000000002, 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 21989 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2399999999999999e-07 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529389+0000", 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3759999999999999e-06 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529398+0000", 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4080000000000004e-06 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529404+0000", 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8919999999999997e-06 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.544686+0000", 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015282662000000001 2024-09-18T00:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.544799+0000", 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011215800000000001 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.545254+0000", 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045577600000000002 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.547172+0000", 2024-09-18T00:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019178999999999999 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550317+0000", 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031449310000000001 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550437+0000", 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119963 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550488+0000", 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0526999999999999e-05 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:21990 3.1d 3:beac871e:::rbd_object_map.117c21a2e20a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.998662868, 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023815820000000001, 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 21990 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529414+0000", 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529417+0000", 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2849999999999999e-06 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529425+0000", 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7740000000000004e-06 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529431+0000", 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8939999999999998e-06 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546090+0000", 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016659706 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546185+0000", 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4889999999999994e-05 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546569+0000", 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038352599999999997 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.547692+0000", 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001123024 2024-09-18T00:58:17.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553127+0000", 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054353630000000003 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553188+0000", 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1247999999999994e-05 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553229+0000", 2024-09-18T00:58:17.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0646999999999999e-05 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26860 3.a 3:523ff797:::rbd_data.117c4c7db1a0.0000000000000039:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.155383832, 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028475341000000001, 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:17.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26860 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:17.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372693+0000", 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0550000000000001e-06 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372699+0000", 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:17.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372714+0000", 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4673e-05 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372722+0000", 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4659999999999999e-06 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372787+0000", 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5288000000000002e-05 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372874+0000", 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6485999999999998e-05 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.377992+0000", 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051177159999999996 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401008+0000", 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02301603 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401026+0000", 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712e-05 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401082+0000", 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5648e-05 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401168+0000", 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5476999999999995e-05 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.656 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.657 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.657 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:17.657 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:17.657 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:17.658 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:17.659 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:17.658+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:17.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:17.761 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:17.758+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:17.790 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:17.791 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:17.791 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:17.791 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:17.805 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:17.858 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:17.859 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:17.862 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:17.858+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:17.875 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:17.891 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:17.892 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:17.908 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:17.909 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:17.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:17.930 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:17.948 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:17.948 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:17.963 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:17.958+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:17.967 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:17.968 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:17.969 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:17.969 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:17.982 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:17.989 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:17.989 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:18.064 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:18.058+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:18.103 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:18.103 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:18.117 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T00:58:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:58:18.167 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:18.158+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:18.201 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T00:58:18.245 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone18 from 0xdf39a16 (overlap 0xc7c3d52) to 0x63e1ea9 2024-09-18T00:58:18.265 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:18.262+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:18.300 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:18.300 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:18.314 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:18.403 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-18T00:58:18.405 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:18.402+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:18.467 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:18.462+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:18.478 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:18.478 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:18.492 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:18.568 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:18.562+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:18.651 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:18.651 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:18.666 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:18.669 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:18.662+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:18.754 INFO:teuthology.orchestra.run.smithi042.stdout:1686 punch from 0x2998526 to 0x299899e, (0x478 bytes) 2024-09-18T00:58:18.761 INFO:teuthology.orchestra.run.smithi042.stdout:1688 punch from 0xae8e24f to 0xae90976, (0x2727 bytes) 2024-09-18T00:58:18.761 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:18.761 INFO:teuthology.orchestra.run.smithi042.stdout:1690 read 0x57bd684 thru 0x57bffe4 (0x2961 bytes) 2024-09-18T00:58:18.762 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:18.762 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:18.762 INFO:teuthology.orchestra.run.smithi042.stdout:1694 punch from 0x3bef4b7 to 0x3bfa366, (0xaeaf bytes) 2024-09-18T00:58:18.764 INFO:teuthology.orchestra.run.smithi042.stdout:1696 write 0xa9798d7 thru 0xa980ea4 (0x75ce bytes) 2024-09-18T00:58:18.768 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:18.762+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:18.782 INFO:teuthology.orchestra.run.smithi042.stdout:1698 read 0x1167f22 thru 0x116aefc (0x2fdb bytes) 2024-09-18T00:58:18.787 INFO:teuthology.orchestra.run.smithi042.stdout:1699 read 0xd9bcb57 thru 0xd9be146 (0x15f0 bytes) 2024-09-18T00:58:18.793 INFO:teuthology.orchestra.run.smithi042.stdout:1700 read 0x8316635 thru 0x831add5 (0x47a1 bytes) 2024-09-18T00:58:18.795 INFO:teuthology.orchestra.run.smithi042.stdout:1702 writesame 0x7357f05 thru 0x735e0c1 data_size 0x1(0x61bd bytes) 2024-09-18T00:58:18.801 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:18.801 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:18.818 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:18.865 INFO:teuthology.orchestra.run.smithi042.stdout:1703 write 0x72654a0 thru 0x726610a (0xc6b bytes) 2024-09-18T00:58:18.869 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:18.862+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:18.871 INFO:teuthology.orchestra.run.smithi042.stdout:1704 punch from 0x6579e4d to 0x657d3d5, (0x3588 bytes) 2024-09-18T00:58:18.871 INFO:teuthology.orchestra.run.smithi042.stdout:1705 writesame 0x8931bf1 thru 0x89351aa data_size 0x2(0x35ba bytes) 2024-09-18T00:58:18.892 INFO:teuthology.orchestra.run.smithi042.stdout:1706 trunc from 0xdf39a16 to 0xb040fe8 2024-09-18T00:58:18.953 INFO:teuthology.orchestra.run.smithi042.stdout:1707 read 0x7c9dd85 thru 0x7ca0e74 (0x30f0 bytes) 2024-09-18T00:58:18.962 INFO:teuthology.orchestra.run.smithi042.stdout:1708 trunc from 0xb040fe8 to 0x6f863f6 2024-09-18T00:58:18.970 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:18.966+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:19.000 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:19.000 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:19.020 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:19.025 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.026 INFO:teuthology.orchestra.run.smithi042.stdout:1710 read 0x106fa7b thru 0x107a427 (0xa9ad bytes) 2024-09-18T00:58:19.028 INFO:teuthology.orchestra.run.smithi042.stdout:1711 write 0x1a02ab3 thru 0x1a0eefd (0xc44b bytes) 2024-09-18T00:58:19.071 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:19.066+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:19.123 INFO:teuthology.orchestra.run.smithi042.stdout:1712 trunc from 0x6f863f6 to 0xfea76a 2024-09-18T00:58:19.163 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:19.163 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:19.169 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T00:58:19.171 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:19.166+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:19.180 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:19.204 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T00:58:19.215 INFO:teuthology.orchestra.run.smithi042.stdout:1713 writesame 0x155d65a thru 0x155fb93 data_size 0x2(0x253a bytes) 2024-09-18T00:58:19.244 INFO:teuthology.orchestra.run.smithi042.stdout:1714 write 0xc2c39df thru 0xc2ce10a (0xa72c bytes) 2024-09-18T00:58:19.249 INFO:teuthology.orchestra.run.smithi042.stdout:1715 read 0x22002d thru 0x22a665 (0xa639 bytes) 2024-09-18T00:58:19.250 INFO:teuthology.orchestra.run.smithi042.stdout:1717 write 0x1fa7c0e thru 0x1fb338f (0xb782 bytes) 2024-09-18T00:58:19.252 INFO:teuthology.orchestra.run.smithi042.stdout:1719 writesame 0x6c7299 thru 0x6cb5b4 data_size 0xa(0x431c bytes) 2024-09-18T00:58:19.255 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.255 INFO:teuthology.orchestra.run.smithi042.stdout:1723 read 0x7c3bd8c thru 0x7c47917 (0xbb8c bytes) 2024-09-18T00:58:19.255 INFO:teuthology.orchestra.run.smithi042.stdout:1724 write 0x4645f84 thru 0x464e7b4 (0x8831 bytes) 2024-09-18T00:58:19.257 INFO:teuthology.orchestra.run.smithi042.stdout:1725 write 0x54edf22 thru 0x54fd977 (0xfa56 bytes) 2024-09-18T00:58:19.263 INFO:teuthology.orchestra.run.smithi042.stdout:1726 write 0x4e204e9 thru 0x4e2329b (0x2db3 bytes) 2024-09-18T00:58:19.266 INFO:teuthology.orchestra.run.smithi042.stdout:1727 punch from 0x5931460 to 0x5933a5b, (0x25fb bytes) 2024-09-18T00:58:19.267 INFO:teuthology.orchestra.run.smithi042.stdout:1728 punch from 0x377ca32 to 0x378b5f3, (0xebc1 bytes) 2024-09-18T00:58:19.267 INFO:teuthology.orchestra.run.smithi042.stdout:1729 write 0x7a143e9 thru 0x7a18001 (0x3c19 bytes) 2024-09-18T00:58:19.268 INFO:teuthology.orchestra.run.smithi042.stdout:1730 punch from 0x24f1cbe to 0x25005b6, (0xe8f8 bytes) 2024-09-18T00:58:19.271 INFO:teuthology.orchestra.run.smithi042.stdout:1731 trunc from 0xc2ce10b to 0xde4c7c6 2024-09-18T00:58:19.272 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:19.266+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:19.279 INFO:teuthology.orchestra.run.smithi042.stdout:1735 read 0xba62e8 thru 0xbaca6f (0x6788 bytes) 2024-09-18T00:58:19.279 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.280 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.298 INFO:teuthology.orchestra.run.smithi042.stdout:1738 trunc from 0xde4c7c6 to 0x2f6405a 2024-09-18T00:58:19.319 INFO:teuthology.orchestra.run.smithi042.stdout:1739 punch from 0x1e7cccd to 0x1e8c0c4, (0xf3f7 bytes) 2024-09-18T00:58:19.319 INFO:teuthology.orchestra.run.smithi042.stdout:1740 punch from 0x290cee3 to 0x2915fbe, (0x90db bytes) 2024-09-18T00:58:19.319 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.319 INFO:teuthology.orchestra.run.smithi042.stdout:1745 read 0x24f17be thru 0x25006f2 (0xef35 bytes) 2024-09-18T00:58:19.343 INFO:teuthology.orchestra.run.smithi042.stdout:1746 write 0xb25d646 thru 0xb263bd7 (0x6592 bytes) 2024-09-18T00:58:19.349 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:19.349 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:19.350 INFO:teuthology.orchestra.run.smithi042.stdout:1747 flatten 2024-09-18T00:58:19.365 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:19.373 INFO:teuthology.orchestra.run.smithi042.stdout:1748 write 0xcae542c thru 0xcaf3351 (0xdf26 bytes) 2024-09-18T00:58:19.380 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.392 INFO:teuthology.orchestra.run.smithi042.stdout:1752 writesame 0xeb72752 thru 0xeb7c98c data_size 0x11(0xa23b bytes) 2024-09-18T00:58:19.393 INFO:teuthology.orchestra.run.smithi042.stdout:1753 punch from 0x4e9a864 to 0x4ea858d, (0xdd29 bytes) 2024-09-18T00:58:19.393 INFO:teuthology.orchestra.run.smithi042.stdout:1754 write 0x7a62527 thru 0x7a6da4f (0xb529 bytes) 2024-09-18T00:58:19.399 INFO:teuthology.orchestra.run.smithi042.stdout:1755 writesame 0x49b4629 thru 0x49b8605 data_size 0x1(0x3fdd bytes) 2024-09-18T00:58:19.409 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.409 INFO:teuthology.orchestra.run.smithi042.stdout:1757 punch from 0xdb5d0df to 0xdb6cee3, (0xfe04 bytes) 2024-09-18T00:58:19.409 INFO:teuthology.orchestra.run.smithi042.stdout:1758 punch from 0x2c02239 to 0x2c09216, (0x6fdd bytes) 2024-09-18T00:58:19.409 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.409 INFO:teuthology.orchestra.run.smithi042.stdout:1760 read 0xcbb2093 thru 0xcbb37f0 (0x175e bytes) 2024-09-18T00:58:19.410 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:19.406+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:19.410 INFO:teuthology.orchestra.run.smithi042.stdout:1762 trunc from 0xeb7c98d to 0xe376190 2024-09-18T00:58:19.428 INFO:teuthology.orchestra.run.smithi042.stdout:1763 trunc from 0xe376190 to 0x851d685 2024-09-18T00:58:19.444 INFO:teuthology.orchestra.run.smithi042.stdout:1764 read 0x329ab44 thru 0x32a9092 (0xe54f bytes) 2024-09-18T00:58:19.445 INFO:teuthology.orchestra.run.smithi042.stdout:1766 punch from 0x7974b32 to 0x7978654, (0x3b22 bytes) 2024-09-18T00:58:19.455 INFO:teuthology.orchestra.run.smithi042.stdout:1767 writesame 0xa4259ad thru 0xa4292a8 data_size 0xe(0x38fc bytes) 2024-09-18T00:58:19.456 INFO:teuthology.orchestra.run.smithi042.stdout:1768 read 0x7630bfe thru 0x763b609 (0xaa0c bytes) 2024-09-18T00:58:19.456 INFO:teuthology.orchestra.run.smithi042.stdout:1770 punch from 0x23ae5a1 to 0x23b6472, (0x7ed1 bytes) 2024-09-18T00:58:19.460 INFO:teuthology.orchestra.run.smithi042.stdout:1771 trunc from 0xa4292a9 to 0xc92ef73 2024-09-18T00:58:19.474 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:19.470+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:19.481 INFO:teuthology.orchestra.run.smithi042.stdout:1772 trunc from 0xc92ef73 to 0x6108714 2024-09-18T00:58:19.523 INFO:teuthology.orchestra.run.smithi042.stdout:1773 writesame 0xec36555 thru 0xec41e64 data_size 0x70(0xb910 bytes) 2024-09-18T00:58:19.523 INFO:teuthology.orchestra.run.smithi042.stdout:1774 read 0xbacd691 thru 0xbadaeb3 (0xd823 bytes) 2024-09-18T00:58:19.524 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.524 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.524 INFO:teuthology.orchestra.run.smithi042.stdout:1777 writesame 0xd7e5fc thru 0xd8b8e7 data_size 0x2(0xd2ec bytes) 2024-09-18T00:58:19.543 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:19.544 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:19.544 INFO:teuthology.orchestra.run.smithi042.stdout:1778 write 0x2be2e11 thru 0x2be5668 (0x2858 bytes) 2024-09-18T00:58:19.546 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.547 INFO:teuthology.orchestra.run.smithi042.stdout:1780 read 0x84c46aa thru 0x84c5d00 (0x1657 bytes) 2024-09-18T00:58:19.548 INFO:teuthology.orchestra.run.smithi042.stdout:1781 writesame 0x66fc5ac thru 0x670c438 data_size 0x1(0xfe8d bytes) 2024-09-18T00:58:19.561 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:19.579 INFO:teuthology.orchestra.run.smithi042.stdout:1782 write 0x3fc2050 thru 0x3fcbc23 (0x9bd4 bytes) 2024-09-18T00:58:19.582 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.582 INFO:teuthology.orchestra.run.smithi042.stdout:1784 write 0x8f75a86 thru 0x8f78db5 (0x3330 bytes) 2024-09-18T00:58:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:1786 write 0x55b6d6f thru 0x55ba835 (0x3ac7 bytes) 2024-09-18T00:58:19.590 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.590 INFO:teuthology.orchestra.run.smithi042.stdout:1790 punch from 0x64b5f58 to 0x64c091a, (0xa9c2 bytes) 2024-09-18T00:58:19.590 INFO:teuthology.orchestra.run.smithi042.stdout:1791 writesame 0x930006c thru 0x9300b4b data_size 0x18(0xae0 bytes) 2024-09-18T00:58:19.594 INFO:teuthology.orchestra.run.smithi042.stdout:1793 read 0x71c4d85 thru 0x71cc6f1 (0x796d bytes) 2024-09-18T00:58:19.594 INFO:teuthology.orchestra.run.smithi042.stdout:1794 read 0x3560523 thru 0x356d79c (0xd27a bytes) 2024-09-18T00:58:19.595 INFO:teuthology.orchestra.run.smithi042.stdout:1795 punch from 0xe5fdc41 to 0xe6018be, (0x3c7d bytes) 2024-09-18T00:58:19.595 INFO:teuthology.orchestra.run.smithi042.stdout:1796 write 0x232af4b thru 0x2330b01 (0x5bb7 bytes) 2024-09-18T00:58:19.597 INFO:teuthology.orchestra.run.smithi042.stdout:1797 punch from 0x63006b to 0x639fbc, (0x9f51 bytes) 2024-09-18T00:58:19.598 INFO:teuthology.orchestra.run.smithi042.stdout:1798 read 0x21c1870 thru 0x21cb426 (0x9bb7 bytes) 2024-09-18T00:58:19.598 INFO:teuthology.orchestra.run.smithi042.stdout:1800 read 0x21a1578 thru 0x21a8734 (0x71bd bytes) 2024-09-18T00:58:19.605 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:19.602+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:19.618 INFO:teuthology.orchestra.run.smithi042.stdout:1801 trunc from 0xec41e65 to 0x6a59b87 2024-09-18T00:58:19.636 INFO:teuthology.orchestra.run.smithi042.stdout:1802 read 0x3bdada1 thru 0x3be0080 (0x52e0 bytes) 2024-09-18T00:58:19.649 INFO:teuthology.orchestra.run.smithi042.stdout:1803 write 0x74a5aec thru 0x74af2c7 (0x97dc bytes) 2024-09-18T00:58:19.651 INFO:teuthology.orchestra.run.smithi042.stdout:1805 punch from 0x50e79ee to 0x50ef47d, (0x7a8f bytes) 2024-09-18T00:58:19.668 INFO:teuthology.orchestra.run.smithi042.stdout:1806 trunc from 0x74af2c8 to 0xdc43820 2024-09-18T00:58:19.675 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:19.670+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:19.681 INFO:teuthology.orchestra.run.smithi042.stdout:1807 punch from 0x34f7e96 to 0x350516a, (0xd2d4 bytes) 2024-09-18T00:58:19.681 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.691 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:19.691 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:19.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:19.715 INFO:teuthology.orchestra.run.smithi042.stdout:1809 trunc from 0xdc43820 to 0xd6c439 2024-09-18T00:58:19.745 INFO:teuthology.orchestra.run.smithi042.stdout:1810 write 0xe819cc thru 0xe8a005 (0x863a bytes) 2024-09-18T00:58:19.767 INFO:teuthology.orchestra.run.smithi042.stdout:1811 writesame 0x7eecc17 thru 0x7ef7386 data_size 0x13(0xa770 bytes) 2024-09-18T00:58:19.768 INFO:teuthology.orchestra.run.smithi042.stdout:1815 punch from 0x53c8980 to 0x53cbc29, (0x32a9 bytes) 2024-09-18T00:58:19.768 INFO:teuthology.orchestra.run.smithi042.stdout:1816 write 0x707a8f8 thru 0x7085116 (0xa81f bytes) 2024-09-18T00:58:19.777 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:19.774+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:19.791 INFO:teuthology.orchestra.run.smithi042.stdout:1817 write 0xb6ad597 thru 0xb6b71f9 (0x9c63 bytes) 2024-09-18T00:58:19.798 INFO:teuthology.orchestra.run.smithi042.stdout:1818 punch from 0x457b35b to 0x457f300, (0x3fa5 bytes) 2024-09-18T00:58:19.799 INFO:teuthology.orchestra.run.smithi042.stdout:1819 read 0x7c25f86 thru 0x7c27d97 (0x1e12 bytes) 2024-09-18T00:58:19.799 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:19.811 INFO:teuthology.orchestra.run.smithi042.stdout:1821 trunc from 0xb6b71fa to 0x3c81496 2024-09-18T00:58:19.856 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:19.856 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:19.859 INFO:teuthology.orchestra.run.smithi042.stdout:1822 write 0x5ec8bd3 thru 0x5ecedcf (0x61fd bytes) 2024-09-18T00:58:19.878 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:19.874+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:19.880 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:19.893 INFO:teuthology.orchestra.run.smithi042.stdout:1825 write 0xcce3b54 thru 0xccef914 (0xbdc1 bytes) 2024-09-18T00:58:19.916 INFO:teuthology.orchestra.run.smithi042.stdout:1826 write 0xe9aa0a4 thru 0xe9b7ac8 (0xda25 bytes) 2024-09-18T00:58:19.925 INFO:teuthology.orchestra.run.smithi042.stdout:1827 writesame 0xa61e740 thru 0xa62a6d9 data_size 0x96(0xbf9a bytes) 2024-09-18T00:58:19.934 INFO:teuthology.orchestra.run.smithi042.stdout:1828 writesame 0x6ba0e75 thru 0x6ba1fb4 data_size 0x17(0x1140 bytes) 2024-09-18T00:58:19.939 INFO:teuthology.orchestra.run.smithi042.stdout:1829 punch from 0x3c77ab3 to 0x3c8449e, (0xc9eb bytes) 2024-09-18T00:58:19.963 INFO:teuthology.orchestra.run.smithi042.stdout:1830 trunc from 0xe9b7ac9 to 0x9bbc72 2024-09-18T00:58:19.979 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:19.974+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:19.994 INFO:teuthology.orchestra.run.smithi042.stdout:1831 read 0x533e27 thru 0x538d2f (0x4f09 bytes) 2024-09-18T00:58:20.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:20.011 INFO:teuthology.orchestra.run.smithi042.stdout:1834 writesame 0x23b17fb thru 0x23bf70a data_size 0x56(0xdf10 bytes) 2024-09-18T00:58:20.027 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:20.028 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:20.035 INFO:teuthology.orchestra.run.smithi042.stdout:1835 write 0x87dfc8c thru 0x87e9c7a (0x9fef bytes) 2024-09-18T00:58:20.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:20.060 INFO:teuthology.orchestra.run.smithi042.stdout:1836 write 0x8d0d1e8 thru 0x8d1bff6 (0xee0f bytes) 2024-09-18T00:58:20.071 INFO:teuthology.orchestra.run.smithi042.stdout:1838 punch from 0x174cddc to 0x175b0d9, (0xe2fd bytes) 2024-09-18T00:58:20.080 INFO:teuthology.orchestra.run.smithi042.stdout:1840 trunc from 0x8d1bff7 to 0x2fe50fd 2024-09-18T00:58:20.080 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:20.074+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:20.135 INFO:teuthology.orchestra.run.smithi042.stdout:1842 writesame 0xb2997b5 thru 0xb2a6078 data_size 0x6(0xc8c4 bytes) 2024-09-18T00:58:20.149 INFO:teuthology.orchestra.run.smithi042.stdout:1843 writesame 0xcef8822 thru 0xcf01658 data_size 0x7(0x8e37 bytes) 2024-09-18T00:58:20.156 INFO:teuthology.orchestra.run.smithi042.stdout:1845 trunc from 0xcf01659 to 0x893d253 2024-09-18T00:58:20.167 INFO:teuthology.orchestra.run.smithi042.stdout:1846 read 0x21ab06b thru 0x21b5a03 (0xa999 bytes) 2024-09-18T00:58:20.168 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.168 INFO:teuthology.orchestra.run.smithi042.stdout:1848 writesame 0x722795e thru 0x72317f8 data_size 0x1(0x9e9b bytes) 2024-09-18T00:58:20.173 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:20.174 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:20.181 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:20.174+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:20.184 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T00:58:20.197 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:20.197 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:20.198 INFO:teuthology.orchestra.run.smithi042.stdout:1849 read 0x4eb20af thru 0x4eb9c83 (0x7bd5 bytes) 2024-09-18T00:58:20.198 INFO:teuthology.orchestra.run.smithi042.stdout:1850 write 0x4bafed3 thru 0x4bb8b26 (0x8c54 bytes) 2024-09-18T00:58:20.201 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:58:20.202 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:20.202 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.136018951, 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.203 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:58:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28571 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16109568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.538271+0000", 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6617125640000001, 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049622299000000002, 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28571 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538271+0000", 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538271+0000", 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538272+0000", 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3599999999999999e-07 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538278+0000", 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538285+0000", 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.46e-06 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538305+0000", 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0545000000000001e-05 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.587723+0000", 2024-09-18T00:58:20.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049417852999999998 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.587808+0000", 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5063999999999997e-05 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.587894+0000", 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5735000000000005e-05 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28572 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16121856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.538314+0000", 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6616697309999999, 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049780243000000002, 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28572 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.214 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538314+0000", 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.538314+0000", 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.541764+0000", 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034495239999999998 2024-09-18T00:58:20.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.541769+0000", 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3539999999999999e-06 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.541784+0000", 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5221999999999999e-05 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.541793+0000", 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4300000000000006e-06 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.587936+0000", 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046143211000000003 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.588015+0000", 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8961999999999994e-05 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.588094+0000", 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9540000000000001e-05 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28590 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16343040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542264+0000", 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6577200449999996, 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049631623, 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28590 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542264+0000", 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542264+0000", 2024-09-18T00:58:20.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542264+0000", 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.75e-07 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542267+0000", 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3879999999999998e-06 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542274+0000", 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.844e-06 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542279+0000", 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1340000000000003e-06 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.591736+0000", 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049457713 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.591816+0000", 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9722999999999995e-05 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.591896+0000", 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9345999999999996e-05 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28591 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16355328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.222 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542287+0000", 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6576974680000003, 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049828273999999999, 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28591 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542287+0000", 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542287+0000", 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542287+0000", 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542289+0000", 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.424e-06 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542306+0000", 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6639000000000001e-05 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542312+0000", 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6659999999999996e-06 2024-09-18T00:58:20.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.591941+0000", 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049629672999999999 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592028+0000", 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6810000000000004e-05 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592115+0000", 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6621999999999998e-05 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28592 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16367616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542320+0000", 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6576637670000007, 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049995825000000001, 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28592 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542320+0000", 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542320+0000", 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542321+0000", 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-07 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542323+0000", 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6759999999999999e-06 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542330+0000", 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9190000000000002e-06 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542336+0000", 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3589999999999999e-06 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592157+0000", 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049821097000000002 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592234+0000", 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7729000000000001e-05 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592316+0000", 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1559999999999998e-05 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28593 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16379904~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.231 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542361+0000", 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6576225969999996, 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050172784999999998, 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28593 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542361+0000", 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542361+0000", 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542362+0000", 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0399999999999996e-07 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542365+0000", 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.864e-06 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542373+0000", 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0979999999999994e-06 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542379+0000", 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7270000000000004e-06 2024-09-18T00:58:20.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592362+0000", 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049982930000000002 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592446+0000", 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4442000000000002e-05 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592534+0000", 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8120000000000003e-05 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28594 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16392192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542387+0000", 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6575973229999992, 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050350209999999999, 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28594 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542387+0000", 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542387+0000", 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542387+0000", 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3200000000000005e-07 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542389+0000", 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0980000000000001e-06 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542403+0000", 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4205e-05 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542411+0000", 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4710000000000001e-06 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592577+0000", 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050165609 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592658+0000", 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1848000000000006e-05 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592737+0000", 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8447000000000003e-05 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28595 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16404480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542419+0000", 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6575652549999997, 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050463569, 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28595 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542419+0000", 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542419+0000", 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542419+0000", 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3599999999999999e-07 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542426+0000", 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9210000000000003e-06 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542433+0000", 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8959999999999997e-06 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542448+0000", 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4701999999999999e-05 2024-09-18T00:58:20.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592780+0000", 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050331819 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592831+0000", 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1533999999999998e-05 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592882+0000", 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1261000000000003e-05 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28596 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16416768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542455+0000", 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6575291829999994, 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050626526999999998, 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28596 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542455+0000", 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542455+0000", 2024-09-18T00:58:20.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542455+0000", 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7100000000000001e-07 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542458+0000", 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1770000000000001e-06 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542464+0000", 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2709999999999998e-06 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542469+0000", 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7500000000000003e-06 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.592926+0000", 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050457098999999998 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593004+0000", 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8147999999999998e-05 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593081+0000", 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7410999999999995e-05 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28597 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16429056~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.249 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542485+0000", 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6574992710000007, 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050727758999999997, 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28597 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542485+0000", 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542485+0000", 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542485+0000", 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-07 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542493+0000", 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542510+0000", 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6135999999999999e-05 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542519+0000", 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.183e-06 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593119+0000", 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050600539 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593166+0000", 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6220999999999997e-05 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593212+0000", 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6896e-05 2024-09-18T00:58:20.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28598 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16441344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.542529+0000", 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6574550390000002, 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050882327999999998, 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28598 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542529+0000", 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.542529+0000", 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.545724+0000", 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031947960000000002 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.545729+0000", 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7579999999999998e-06 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.545742+0000", 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2877999999999999e-05 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.545750+0000", 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0080000000000002e-06 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593253+0000", 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.047502731999999999 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593334+0000", 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1162999999999995e-05 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.593411+0000", 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6993e-05 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28614 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16637952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546234+0000", 2024-09-18T00:58:20.258 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6537503309999995, 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049626101999999998, 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28614 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546234+0000", 2024-09-18T00:58:20.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546234+0000", 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546234+0000", 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546237+0000", 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5129999999999999e-06 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546252+0000", 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5727999999999999e-05 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546259+0000", 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2659999999999998e-06 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595779+0000", 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049520284999999997 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595818+0000", 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9542999999999997e-05 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595860+0000", 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1337000000000001e-05 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28615 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16650240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546267+0000", 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6537172059999996, 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049708853999999997, 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28615 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546267+0000", 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546267+0000", 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546267+0000", 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9299999999999998e-07 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546270+0000", 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7439999999999999e-06 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546278+0000", 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6620000000000005e-06 2024-09-18T00:58:20.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546283+0000", 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1290000000000004e-06 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595891+0000", 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049608273000000001 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595929+0000", 2024-09-18T00:58:20.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7877999999999999e-05 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.595976+0000", 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6675000000000003e-05 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28616 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16662528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546291+0000", 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6536934809999995, 2024-09-18T00:58:20.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049808481000000002, 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28616 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546291+0000", 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546291+0000", 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546291+0000", 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8800000000000003e-07 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546293+0000", 2024-09-18T00:58:20.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3429999999999998e-06 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546308+0000", 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4833e-05 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546314+0000", 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0689999999999998e-06 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596018+0000", 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049703658999999997 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596058+0000", 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0170999999999999e-05 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596099+0000", 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0917999999999997e-05 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.271 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28617 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16674816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546346+0000", 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6536376520000005, 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049862966000000002, 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28617 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546346+0000", 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546346+0000", 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546347+0000", 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5899999999999996e-07 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546350+0000", 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9919999999999999e-06 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546366+0000", 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5698000000000001e-05 2024-09-18T00:58:20.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546372+0000", 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5649999999999998e-06 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596128+0000", 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049756250000000002 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596166+0000", 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7586999999999998e-05 2024-09-18T00:58:20.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596209+0000", 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3315000000000001e-05 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28618 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16687104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546382+0000", 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6536021180000002, 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049948445000000001, 2024-09-18T00:58:20.276 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28618 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546382+0000", 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546382+0000", 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546382+0000", 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546385+0000", 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6570000000000001e-06 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546394+0000", 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1220000000000001e-06 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546400+0000", 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6640000000000003e-06 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596250+0000", 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049849656999999999 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596289+0000", 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9681000000000002e-05 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596330+0000", 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1050000000000002e-05 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.283 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28619 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16699392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546412+0000", 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6535723410000003, 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050029523999999999, 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.285 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.285 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28619 2024-09-18T00:58:20.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.285 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546412+0000", 2024-09-18T00:58:20.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546412+0000", 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546412+0000", 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-07 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546421+0000", 2024-09-18T00:58:20.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5110000000000003e-06 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546428+0000", 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1930000000000004e-06 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546433+0000", 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2059999999999998e-06 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596360+0000", 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049926816999999998 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596398+0000", 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7712000000000001e-05 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596441+0000", 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3616999999999999e-05 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.288 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28620 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16711680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546465+0000", 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6535188559999998, 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050094769999999997, 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28620 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546465+0000", 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546465+0000", 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546466+0000", 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0999999999999998e-07 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546469+0000", 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8720000000000002e-06 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546477+0000", 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0160000000000008e-06 2024-09-18T00:58:20.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546483+0000", 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9529999999999996e-06 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596482+0000", 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049998919000000003 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596520+0000", 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8359000000000003e-05 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596560+0000", 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9941000000000003e-05 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:28621 3.c 3:3733b31e:::rbd_data.117c86759014.0000000000000005:12 [read 16723968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:11.546492+0000", 2024-09-18T00:58:20.293 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6534920139999993, 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050179060999999997, 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28621 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546492+0000", 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.546492+0000", 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.549801+0000", 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033093630000000001 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.549807+0000", 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8170000000000004e-06 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.549818+0000", 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1139e-05 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.549828+0000", 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9389999999999992e-06 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596590+0000", 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046761745 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596628+0000", 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7601999999999997e-05 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:11.596671+0000", 2024-09-18T00:58:20.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3455999999999999e-05 2024-09-18T00:58:20.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.298 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.298 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.298 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:20.299 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.299 INFO:teuthology.orchestra.run.smithi042.stdout:1853 read 0x3545da7 thru 0x354d662 (0x78bc bytes) 2024-09-18T00:58:20.300 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T00:58:20.300 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.300 INFO:teuthology.orchestra.run.smithi042.stdout:1855 writesame 0x431aa thru 0x45f08 data_size 0x17(0x2d5f bytes) 2024-09-18T00:58:20.301 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:20.301 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:20.301 INFO:teuthology.orchestra.run.smithi042.stdout:1856 trunc from 0x893d253 to 0x6e91361 2024-09-18T00:58:20.302 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:20.303 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:20.303 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:20.304 INFO:teuthology.orchestra.run.smithi042.stdout:1858 punch from 0xc06f17 to 0xc09baa, (0x2c93 bytes) 2024-09-18T00:58:20.304 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.304 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.304 INFO:teuthology.orchestra.run.smithi042.stdout:1861 write 0x3d4aa3b thru 0x3d525ab (0x7b71 bytes) 2024-09-18T00:58:20.304 INFO:teuthology.orchestra.run.smithi042.stdout:1863 read 0x5f087e4 thru 0x5f0d19f (0x49bc bytes) 2024-09-18T00:58:20.304 INFO:teuthology.orchestra.run.smithi042.stdout:1864 write 0x31d71b9 thru 0x31e30a5 (0xbeed bytes) 2024-09-18T00:58:20.304 INFO:teuthology.orchestra.run.smithi042.stdout:1865 read 0x35b66e thru 0x365513 (0x9ea6 bytes) 2024-09-18T00:58:20.304 INFO:teuthology.orchestra.run.smithi042.stdout:1867 write 0x7c56ca thru 0x7d2b9e (0xd4d5 bytes) 2024-09-18T00:58:20.305 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:20.305 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:20.305 INFO:teuthology.orchestra.run.smithi042.stdout:1869 write 0xd7f97a4 thru 0xd801dcc (0x8629 bytes) 2024-09-18T00:58:20.305 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:20.306 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:20.306 INFO:teuthology.orchestra.run.smithi042.stdout:1870 write 0x9bdb8ec thru 0x9be8742 (0xce57 bytes) 2024-09-18T00:58:20.306 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.307 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:20.307 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:20.308 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:20.294+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:20.309 INFO:teuthology.orchestra.run.smithi042.stdout:1873 write 0xe837c7f thru 0xe84409a (0xc41c bytes) 2024-09-18T00:58:20.321 INFO:teuthology.orchestra.run.smithi042.stdout:1875 punch from 0xad11bc6 to 0xad1c83e, (0xac78 bytes) 2024-09-18T00:58:20.321 INFO:teuthology.orchestra.run.smithi042.stdout:1877 write 0x965996b thru 0x965df44 (0x45da bytes) 2024-09-18T00:58:20.326 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.326 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.326 INFO:teuthology.orchestra.run.smithi042.stdout:1882 write 0xd0b83b5 thru 0xd0bbce2 (0x392e bytes) 2024-09-18T00:58:20.328 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:20.328 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:20.328 INFO:teuthology.orchestra.run.smithi042.stdout:1883 read 0xc809af6 thru 0xc819140 (0xf64b bytes) 2024-09-18T00:58:20.329 INFO:teuthology.orchestra.run.smithi042.stdout:1884 read 0xbe9206e thru 0xbe9b93d (0x98d0 bytes) 2024-09-18T00:58:20.329 INFO:teuthology.orchestra.run.smithi042.stdout:1886 read 0x9593664 thru 0x9598ec6 (0x5863 bytes) 2024-09-18T00:58:20.330 INFO:teuthology.orchestra.run.smithi042.stdout:1887 read 0x5d758aa thru 0x5d84917 (0xf06e bytes) 2024-09-18T00:58:20.331 INFO:teuthology.orchestra.run.smithi042.stdout:1888 read 0x15d81af thru 0x15d8928 (0x77a bytes) 2024-09-18T00:58:20.332 INFO:teuthology.orchestra.run.smithi042.stdout:1889 write 0xd99d1c0 thru 0xd9a8476 (0xb2b7 bytes) 2024-09-18T00:58:20.352 INFO:teuthology.orchestra.run.smithi042.stdout:1890 trunc from 0xe84409b to 0x7380cfe 2024-09-18T00:58:20.378 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.379 INFO:teuthology.orchestra.run.smithi042.stdout:1892 punch from 0xb9957d to 0xba0a01, (0x7484 bytes) 2024-09-18T00:58:20.379 INFO:teuthology.orchestra.run.smithi042.stdout:1894 write 0x3819d34 thru 0x381e620 (0x48ed bytes) 2024-09-18T00:58:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:1895 writesame 0x8b559eb thru 0x8b5df19 data_size 0x5(0x852f bytes) 2024-09-18T00:58:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.396 INFO:teuthology.orchestra.run.smithi042.stdout:1897 read 0xb18fd2 thru 0xb2598c (0xc9bb bytes) 2024-09-18T00:58:20.401 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:20.394+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:20.410 INFO:teuthology.orchestra.run.smithi042.stdout:1898 writesame 0x9e02e16 thru 0x9e07faa data_size 0x1(0x5195 bytes) 2024-09-18T00:58:20.423 INFO:teuthology.orchestra.run.smithi042.stdout:1900 trunc from 0x9e07fab to 0x262ae4f 2024-09-18T00:58:20.453 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:20.454 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:20.472 INFO:teuthology.orchestra.run.smithi042.stdout:1902 writesame 0x98e5277 thru 0x98ea973 data_size 0xd(0x56fd bytes) 2024-09-18T00:58:20.472 INFO:teuthology.orchestra.run.smithi042.stdout:1903 write 0x79affc4 thru 0x79bc831 (0xc86e bytes) 2024-09-18T00:58:20.474 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:1906 writesame 0x6b1ef40 thru 0x6b233f4 data_size 0x29(0x44b5 bytes) 2024-09-18T00:58:20.487 INFO:teuthology.orchestra.run.smithi042.stdout:1908 read 0x19f018 thru 0x1a0ede (0x1ec7 bytes) 2024-09-18T00:58:20.488 INFO:teuthology.orchestra.run.smithi042.stdout:1909 punch from 0x7ae3a9f to 0x7af2b6b, (0xf0cc bytes) 2024-09-18T00:58:20.488 INFO:teuthology.orchestra.run.smithi042.stdout:1911 read 0x3e9d2db thru 0x3ea4e3a (0x7b60 bytes) 2024-09-18T00:58:20.489 INFO:teuthology.orchestra.run.smithi042.stdout:1912 read 0x7c44078 thru 0x7c49a5f (0x59e8 bytes) 2024-09-18T00:58:20.501 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:20.498+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:20.502 INFO:teuthology.orchestra.run.smithi042.stdout:1913 write 0xabe51e9 thru 0xabeb7d7 (0x65ef bytes) 2024-09-18T00:58:20.509 INFO:teuthology.orchestra.run.smithi042.stdout:1914 punch from 0x221ef96 to 0x2222838, (0x38a2 bytes) 2024-09-18T00:58:20.509 INFO:teuthology.orchestra.run.smithi042.stdout:1917 read 0x58a1d75 thru 0x58ac13a (0xa3c6 bytes) 2024-09-18T00:58:20.525 INFO:teuthology.orchestra.run.smithi042.stdout:1918 writesame 0xcbc471e thru 0xcbc5ea8 data_size 0x29(0x178b bytes) 2024-09-18T00:58:20.525 INFO:teuthology.orchestra.run.smithi042.stdout:1919 write 0xb158c78 thru 0xb165119 (0xc4a2 bytes) 2024-09-18T00:58:20.534 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.537 INFO:teuthology.orchestra.run.smithi042.stdout:1921 trunc from 0xcbc5ea9 to 0xe69cd13 2024-09-18T00:58:20.548 INFO:teuthology.orchestra.run.smithi042.stdout:1923 writesame 0xca7d1bc thru 0xca81b92 data_size 0x1(0x49d7 bytes) 2024-09-18T00:58:20.579 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.580 INFO:teuthology.orchestra.run.smithi042.stdout:1925 punch from 0x7eb3cc3 to 0x7eb6029, (0x2366 bytes) 2024-09-18T00:58:20.580 INFO:teuthology.orchestra.run.smithi042.stdout:1926 write 0x5affac7 thru 0x5b0a3b8 (0xa8f2 bytes) 2024-09-18T00:58:20.587 INFO:teuthology.orchestra.run.smithi042.stdout:1927 writesame 0x55a93f5 thru 0x55b8b25 data_size 0x1(0xf731 bytes) 2024-09-18T00:58:20.602 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:20.598+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:20.608 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:20.608 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:20.608 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:20.608 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:20.608 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:20.617 INFO:teuthology.orchestra.run.smithi042.stdout:1928 read 0x97e8273 thru 0x97edf63 (0x5cf1 bytes) 2024-09-18T00:58:20.618 INFO:teuthology.orchestra.run.smithi042.stdout:1929 punch from 0x69b2fe5 to 0x69b75d2, (0x45ed bytes) 2024-09-18T00:58:20.618 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.618 INFO:teuthology.orchestra.run.smithi042.stdout:1931 write 0xa51b012 thru 0xa51c3fa (0x13e9 bytes) 2024-09-18T00:58:20.625 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:20.642 INFO:teuthology.orchestra.run.smithi042.stdout:1932 trunc from 0xe69cd13 to 0x1ed6904 2024-09-18T00:58:20.672 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.672 INFO:teuthology.orchestra.run.smithi042.stdout:1934 read 0x116745d thru 0x117711e (0xfcc2 bytes) 2024-09-18T00:58:20.693 INFO:teuthology.orchestra.run.smithi042.stdout:1938 write 0x5ffe06d thru 0x6003784 (0x5718 bytes) 2024-09-18T00:58:20.699 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.699 INFO:teuthology.orchestra.run.smithi042.stdout:1944 write 0x260a840 thru 0x2610b96 (0x6357 bytes) 2024-09-18T00:58:20.703 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:20.698+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:20.726 INFO:teuthology.orchestra.run.smithi042.stdout:1945 writesame 0xd5e5441 thru 0xd5f3c76 data_size 0x1(0xe836 bytes) 2024-09-18T00:58:20.726 INFO:teuthology.orchestra.run.smithi042.stdout:1946 writesame 0xa150b29 thru 0xa15148c data_size 0x2(0x964 bytes) 2024-09-18T00:58:20.732 INFO:teuthology.orchestra.run.smithi042.stdout:1947 read 0xd254521 thru 0xd263576 (0xf056 bytes) 2024-09-18T00:58:20.751 INFO:teuthology.orchestra.run.smithi042.stdout:1948 writesame 0xe93ba7c thru 0xe945289 data_size 0x1(0x980e bytes) 2024-09-18T00:58:20.751 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.765 INFO:teuthology.orchestra.run.smithi042.stdout:1954 trunc from 0xe94528a to 0x7217a9b 2024-09-18T00:58:20.770 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:58:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 57.478733796999997, 2024-09-18T00:58:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:58:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:58:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:58:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7423 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.282435247000002, 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045296661000000002, 2024-09-18T00:58:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7423 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485621+0000", 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485627+0000", 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.156e-06 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485633+0000", 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789999999999999e-06 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530788+0000", 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045155001 2024-09-18T00:58:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530853+0000", 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5634000000000002e-05 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530915+0000", 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2022999999999999e-05 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7424 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1146880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.282414256999999, 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045419355000000002, 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7424 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:20.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485642+0000", 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0640000000000001e-06 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485649+0000", 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485654+0000", 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2329999999999997e-06 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530941+0000", 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045286602000000002 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531004+0000", 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3455999999999997e-05 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531059+0000", 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4771000000000001e-05 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.282392743999999, 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:20.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:20.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:20.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:58:20.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.282371306999998, 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:20.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.282275980000001, 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:20.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.282250541000003, 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:58:20.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.282228017999998, 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.805 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:20.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.282204577999998, 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.809 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:20.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:58:20.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.813 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.282180768000003, 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:20.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:58:20.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.282158402999997, 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:58:20.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.282131786000001, 2024-09-18T00:58:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:58:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:58:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:58:20.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:58:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:58:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:58:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:58:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.277980014999997, 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:20.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.277959271999997, 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.837 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.277913707000003, 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:58:20.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.277887366000002, 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:58:20.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.277864696000002, 2024-09-18T00:58:20.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:58:20.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:58:20.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.277846322999999, 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:58:20.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.277826535000003, 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:58:20.855 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:20.859 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.277799665000003, 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:20.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:20.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:58:20.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:20.864 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:20.864 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:20.864 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:20.866 INFO:teuthology.orchestra.run.smithi042.stdout:1955 read 0x594bd69 thru 0x595b4dd (0xf775 bytes) 2024-09-18T00:58:20.866 INFO:teuthology.orchestra.run.smithi042.stdout:1956 punch from 0x682caa6 to 0x683c606, (0xfb60 bytes) 2024-09-18T00:58:20.866 INFO:teuthology.orchestra.run.smithi042.stdout:1957 read 0xf92abb thru 0xf9c1c9 (0x970f bytes) 2024-09-18T00:58:20.866 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.866 INFO:teuthology.orchestra.run.smithi042.stdout:1959 write 0x2ec3614 thru 0x2ec5368 (0x1d55 bytes) 2024-09-18T00:58:20.866 INFO:teuthology.orchestra.run.smithi042.stdout:1960 write 0xd0c4c1f thru 0xd0c85cb (0x39ad bytes) 2024-09-18T00:58:20.866 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.866 INFO:teuthology.orchestra.run.smithi042.stdout:1964 read 0x12a8c38 thru 0x12b2590 (0x9959 bytes) 2024-09-18T00:58:20.866 INFO:teuthology.orchestra.run.smithi042.stdout:1965 write 0xedfc0f4 thru 0xedfc6e1 (0x5ee bytes) 2024-09-18T00:58:20.866 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.867 INFO:teuthology.orchestra.run.smithi042.stdout:1969 writesame 0x6bef149 thru 0x6bf282c data_size 0x6(0x36e4 bytes) 2024-09-18T00:58:20.867 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:20.822+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:20.868 INFO:teuthology.orchestra.run.smithi042.stdout:1970 read 0xaa228b7 thru 0xaa2d50a (0xac54 bytes) 2024-09-18T00:58:20.868 INFO:teuthology.orchestra.run.smithi042.stdout:1972 write 0x245bec5 thru 0x246b992 (0xface bytes) 2024-09-18T00:58:20.871 INFO:teuthology.orchestra.run.smithi042.stdout:1973 write 0x71b9c17 thru 0x71c0c53 (0x703d bytes) 2024-09-18T00:58:20.879 INFO:teuthology.orchestra.run.smithi042.stdout:1974 read 0x345b462 thru 0x346349e (0x803d bytes) 2024-09-18T00:58:20.880 INFO:teuthology.orchestra.run.smithi042.stdout:1976 read 0x8038f64 thru 0x803e93b (0x59d8 bytes) 2024-09-18T00:58:20.880 INFO:teuthology.orchestra.run.smithi042.stdout:1978 write 0xb340196 thru 0xb344360 (0x41cb bytes) 2024-09-18T00:58:20.883 INFO:teuthology.orchestra.run.smithi042.stdout:1979 write 0xbbea4e3 thru 0xbbf9d54 (0xf872 bytes) 2024-09-18T00:58:20.889 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.889 INFO:teuthology.orchestra.run.smithi042.stdout:1982 read 0x72d2deb thru 0x72e1933 (0xeb49 bytes) 2024-09-18T00:58:20.890 INFO:teuthology.orchestra.run.smithi042.stdout:1983 write 0x3d113a1 thru 0x3d1345c (0x20bc bytes) 2024-09-18T00:58:20.896 INFO:teuthology.orchestra.run.smithi042.stdout:1984 read 0x4f07c3 thru 0x4faad3 (0xa311 bytes) 2024-09-18T00:58:20.896 INFO:teuthology.orchestra.run.smithi042.stdout:1985 read 0xde4d974 thru 0xde53bf3 (0x6280 bytes) 2024-09-18T00:58:20.896 INFO:teuthology.orchestra.run.smithi042.stdout:1987 write 0x708cb4a thru 0x708e0b7 (0x156e bytes) 2024-09-18T00:58:20.902 INFO:teuthology.orchestra.run.smithi042.stdout:1989 writesame 0x94bcd02 thru 0x94c0d61 data_size 0x28(0x4060 bytes) 2024-09-18T00:58:20.912 INFO:teuthology.orchestra.run.smithi042.stdout:1990 read 0xba9da13 thru 0xbaa5b5c (0x814a bytes) 2024-09-18T00:58:20.912 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.912 INFO:teuthology.orchestra.run.smithi042.stdout:1993 read 0x2787a82 thru 0x2792b30 (0xb0af bytes) 2024-09-18T00:58:20.913 INFO:teuthology.orchestra.run.smithi042.stdout:1994 read 0xe4ba3dc thru 0xe4bcc49 (0x286e bytes) 2024-09-18T00:58:20.913 INFO:teuthology.orchestra.run.smithi042.stdout:1995 read 0x70d1383 thru 0x70d893c (0x75ba bytes) 2024-09-18T00:58:20.929 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:20.922+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:20.934 INFO:teuthology.orchestra.run.smithi042.stdout:1997 trunc from 0xedfc6e2 to 0x3473ca6 2024-09-18T00:58:20.965 INFO:teuthology.orchestra.run.smithi042.stdout:1998 writesame 0x218eb90 thru 0x2194691 data_size 0x21(0x5b02 bytes) 2024-09-18T00:58:20.968 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:20.968 INFO:teuthology.orchestra.run.smithi042.stdout:2002 read 0x243bf8f thru 0x244a08c (0xe0fe bytes) 2024-09-18T00:58:20.969 INFO:teuthology.orchestra.run.smithi042.stdout:2003 punch from 0x1d89cd0 to 0x1d92abb, (0x8deb bytes) 2024-09-18T00:58:20.974 INFO:teuthology.orchestra.run.smithi042.stdout:2004 trunc from 0x3473ca6 to 0x67fa9cc 2024-09-18T00:58:20.986 INFO:teuthology.orchestra.run.smithi042.stdout:2005 read 0x1fa5edb thru 0x1fabf4a (0x6070 bytes) 2024-09-18T00:58:21.000 INFO:teuthology.orchestra.run.smithi042.stdout:2006 write 0x74a7ced thru 0x74a814b (0x45f bytes) 2024-09-18T00:58:21.004 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:21.004 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:21.004 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:21.004 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:21.008 INFO:teuthology.orchestra.run.smithi042.stdout:2007 read 0x3036fd6 thru 0x303f180 (0x81ab bytes) 2024-09-18T00:58:21.009 INFO:teuthology.orchestra.run.smithi042.stdout:2008 punch from 0x6537233 to 0x65468a7, (0xf674 bytes) 2024-09-18T00:58:21.009 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:21.012 INFO:teuthology.orchestra.run.smithi042.stdout:2010 trunc from 0x74a814c to 0x5583281 2024-09-18T00:58:21.023 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:21.036 INFO:teuthology.orchestra.run.smithi042.stdout:2012 trunc from 0x5583281 to 0x16054a8 2024-09-18T00:58:21.036 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:21.030+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:21.062 INFO:teuthology.orchestra.run.smithi042.stdout:2013 read 0x577a62 thru 0x587458 (0xf9f7 bytes) 2024-09-18T00:58:21.063 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:21.085 INFO:teuthology.orchestra.run.smithi042.stdout:2016 write 0x723e81d thru 0x7243d08 (0x54ec bytes) 2024-09-18T00:58:21.120 INFO:teuthology.orchestra.run.smithi042.stdout:2017 write 0xe4004a3 thru 0xe40be67 (0xb9c5 bytes) 2024-09-18T00:58:21.129 INFO:teuthology.orchestra.run.smithi042.stdout:2018 write 0x55f74cf thru 0x560671a (0xf24c bytes) 2024-09-18T00:58:21.130 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:21.126+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:21.146 INFO:teuthology.orchestra.run.smithi042.stdout:2019 trunc from 0xe40be68 to 0x9f2577a 2024-09-18T00:58:21.174 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:21.174 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:21.174 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:21.174 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:21.174 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:21.186 INFO:teuthology.orchestra.run.smithi042.stdout:2022 write 0xa4d19d5 thru 0xa4df20c (0xd838 bytes) 2024-09-18T00:58:21.189 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:21.194 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:58:21.197 INFO:teuthology.orchestra.run.smithi042.stdout:2025 punch from 0x1aee566 to 0x1af0567, (0x2001 bytes) 2024-09-18T00:58:21.197 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T00:58:21.217 INFO:teuthology.orchestra.run.smithi042.stdout:2027 write 0xc297007 thru 0xc2a6443 (0xf43d bytes) 2024-09-18T00:58:21.226 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T00:58:21.228 INFO:teuthology.orchestra.run.smithi042.stdout:2028 read 0x7629c9e thru 0x762c5ed (0x2950 bytes) 2024-09-18T00:58:21.229 INFO:teuthology.orchestra.run.smithi042.stdout:2029 write 0xaa67572 thru 0xaa6b95f (0x43ee bytes) 2024-09-18T00:58:21.231 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:21.226+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:21.232 INFO:teuthology.orchestra.run.smithi042.stdout:2030 writesame 0x21a5eaf thru 0x21b0a99 data_size 0x1(0xabeb bytes) 2024-09-18T00:58:21.269 INFO:teuthology.orchestra.run.smithi042.stdout:2031 trunc from 0xc2a6444 to 0x169c76a 2024-09-18T00:58:21.299 INFO:teuthology.orchestra.run.smithi042.stdout:2032 punch from 0x119794 to 0x124b34, (0xb3a0 bytes) 2024-09-18T00:58:21.308 INFO:teuthology.orchestra.run.smithi042.stdout:2034 trunc from 0x169c76a to 0x65b4631 2024-09-18T00:58:21.330 INFO:teuthology.orchestra.run.smithi042.stdout:2036 writesame 0x7cb3a3f thru 0x7cb7e20 data_size 0x1(0x43e2 bytes) 2024-09-18T00:58:21.330 INFO:teuthology.orchestra.run.smithi042.stdout:2037 write 0x5e4369d thru 0x5e5076e (0xd0d2 bytes) 2024-09-18T00:58:21.332 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:21.332 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.333 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.252940915000003, 2024-09-18T00:58:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:58:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:58:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.335 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:58:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:58:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:58:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:58:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:58:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:58:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:58:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:58:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:58:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:58:21.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:58:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:58:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:58:21.343 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.252011324000001, 2024-09-18T00:58:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:58:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.346 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:58:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:21.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:21.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:58:21.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:58:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:58:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:58:21.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:58:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:58:21.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:58:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:58:21.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:58:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:58:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:58:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:58:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:58:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:58:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:21.356 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.251997264000003, 2024-09-18T00:58:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:58:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:58:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:58:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:58:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:58:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:58:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:58:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:58:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:58:21.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:58:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:58:21.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:58:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:58:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:58:21.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:58:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:58:21.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.367 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.250706688999998, 2024-09-18T00:58:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:58:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.368 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:58:21.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:21.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:58:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:58:21.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:58:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:58:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:58:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:58:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:58:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:58:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:58:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:58:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:58:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:58:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:58:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:58:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.250686217000002, 2024-09-18T00:58:21.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:58:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:58:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:58:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:58:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:58:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:58:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:58:21.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:58:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:58:21.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:58:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:58:21.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:58:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:58:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:58:21.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:58:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:58:21.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.390 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.247891764000002, 2024-09-18T00:58:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:58:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:58:21.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:21.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:58:21.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:58:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:58:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:58:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:58:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:58:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:58:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:58:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:58:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:58:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:58:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:58:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:58:21.400 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.247876580000003, 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.401 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.405 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.247853634000002, 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:21.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:58:21.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.243675994999997, 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.414 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.243654778, 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:58:21.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.243633395000003, 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:58:21.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.239733541, 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:58:21.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:58:21.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.239719340000001, 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:58:21.428 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:58:21.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.432 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.239701302999997, 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.239687236999998, 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:58:21.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.441 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.235661886000003, 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:58:21.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:21.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.235647874000001, 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.450 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.235633909999997, 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:58:21.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:58:21.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.473775955999997, 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:21.455 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:58:21.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:58:21.459 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.348299584999999, 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:58:21.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:58:21.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:58:21.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:58:21.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:58:21.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.465 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.465 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.465 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:21.466 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:21.326+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:21.466 INFO:teuthology.orchestra.run.smithi042.stdout:2038 writesame 0x3e04242 thru 0x3e13f00 data_size 0x1(0xfcbf bytes) 2024-09-18T00:58:21.467 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:2039 read 0x1055f45 thru 0x105bc3b (0x5cf7 bytes) 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:2044 write 0x54a1990 thru 0x54a805e (0x66cf bytes) 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:2046 write 0x267cf6d thru 0x267dfca (0x105e bytes) 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:2048 punch from 0x3f432b5 to 0x3f48642, (0x538d bytes) 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:2049 write 0x262d71f thru 0x262e93f (0x1221 bytes) 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:2050 write 0xb3094d5 thru 0xb3129d0 (0x94fc bytes) 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:2051 writesame 0xa7b913e thru 0xa7c35e2 data_size 0x1(0xa4a5 bytes) 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:2053 punch from 0x9def4b2 to 0x9df55c7, (0x6115 bytes) 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:2055 read 0x2094888 thru 0x209be01 (0x757a bytes) 2024-09-18T00:58:21.469 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:21.470 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:21.462+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:21.471 INFO:teuthology.orchestra.run.smithi042.stdout:2058 write 0xc3a6111 thru 0xc3b5f2c (0xfe1c bytes) 2024-09-18T00:58:21.480 INFO:teuthology.orchestra.run.smithi042.stdout:2059 write 0x893f09 thru 0x899aab (0x5ba3 bytes) 2024-09-18T00:58:21.484 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:21.484 INFO:teuthology.orchestra.run.smithi042.stdout:2062 clone 20 order 22 su 16384 sc 7 2024-09-18T00:58:21.568 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:21.562+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:21.587 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:21.587 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:21.587 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:21.587 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:21.601 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:21.690 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:21.686+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:21.770 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:21.762+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:21.785 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:21.871 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:21.866+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:21.950 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:21.950 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:21.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:21.950 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:21.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.950 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:58:21.950 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:21.950 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 54.632126552000003, 2024-09-18T00:58:21.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101264911, 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4226", 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2155402750", 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313760+0000", 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.612e-06 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313768+0000", 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9249999999999995e-06 2024-09-18T00:58:21.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313786+0000", 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8253999999999998e-05 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313803+0000", 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313935+0000", 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013180399999999999 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313954+0000", 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8768999999999999e-05 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.411874+0000", 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.097920646 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412292+0000", 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041777699999999998 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412675+0000", 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038262300000000001 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414019+0000", 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013436699999999999 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414872+0000", 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085320999999999997 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414955+0000", 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3405999999999994e-05 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.415023+0000", 2024-09-18T00:58:21.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7825999999999997e-05 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3740 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.078321826, 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020503747999999999, 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.957 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3740 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:21.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867563+0000", 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867566+0000", 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3159999999999999e-06 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867571+0000", 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9240000000000004e-06 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867576+0000", 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8210000000000001e-06 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887847+0000", 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020270468 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.887952+0000", 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010541599999999999 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888067+0000", 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000114391 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3741 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.078301947, 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020744149999999999, 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3741 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.962 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867583+0000", 2024-09-18T00:58:21.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0800000000000001e-07 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867585+0000", 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9800000000000001e-06 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867591+0000", 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3269999999999997e-06 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867597+0000", 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0750000000000003e-06 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888108+0000", 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020510589999999999 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888212+0000", 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104252 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888327+0000", 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011461800000000001 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3742 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.078280415999998, 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020983208999999999, 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.966 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3742 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867604+0000", 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867605+0000", 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5100000000000001e-07 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867613+0000", 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867620+0000", 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.793e-06 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867625+0000", 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2619999999999998e-06 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888362+0000", 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020736747999999999 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888469+0000", 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000106969 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888587+0000", 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118904 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3743 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.970 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.078252822000003, 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021178947, 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3743 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867632+0000", 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7800000000000002e-07 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867634+0000", 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.956e-06 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867640+0000", 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0769999999999996e-06 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867645+0000", 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5619999999999997e-06 2024-09-18T00:58:21.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888622+0000", 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020977554999999998 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888715+0000", 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2905000000000006e-05 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888811+0000", 2024-09-18T00:58:21.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5513999999999994e-05 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3744 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.078233543000003, 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021391389, 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3744 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867651+0000", 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8299999999999998e-07 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867653+0000", 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7990000000000001e-06 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867659+0000", 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5999999999999997e-06 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867664+0000", 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8910000000000003e-06 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888846+0000", 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021181867 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.888951+0000", 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000105837 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889042+0000", 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1012000000000002e-05 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3745 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.979 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.078212290000003, 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021600477999999999, 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3745 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:21.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867672+0000", 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867673+0000", 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2599999999999998e-07 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867675+0000", 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0499999999999999e-06 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867682+0000", 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0140000000000001e-06 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867687+0000", 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9790000000000001e-06 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889079+0000", 2024-09-18T00:58:21.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021392347999999999 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889182+0000", 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000102712 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889273+0000", 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1049000000000002e-05 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3746 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.078190702999997, 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021835075999999998, 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3746 2024-09-18T00:58:21.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867694+0000", 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6100000000000002e-07 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867701+0000", 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867709+0000", 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7999999999999999e-06 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867715+0000", 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.699e-06 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889307+0000", 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021592239999999999 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889410+0000", 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103093 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889529+0000", 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011872299999999999 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3747 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.078160554999997, 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022084972000000001, 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3747 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867724+0000", 2024-09-18T00:58:21.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867725+0000", 2024-09-18T00:58:21.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5199999999999996e-07 2024-09-18T00:58:21.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867735+0000", 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7790000000000007e-06 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867743+0000", 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1389999999999995e-06 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867748+0000", 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7209999999999999e-06 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889588+0000", 2024-09-18T00:58:21.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021839447000000001 2024-09-18T00:58:21.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889706+0000", 2024-09-18T00:58:21.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117748 2024-09-18T00:58:21.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889809+0000", 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000103486 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:3748 3.13 3:cf08df90:::rbd_data.117c7eee262f.000000000000008b:4 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.078125376000003, 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022206404999999999, 2024-09-18T00:58:21.993 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3748 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867759+0000", 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867760+0000", 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-07 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867762+0000", 2024-09-18T00:58:21.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6079999999999998e-06 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867769+0000", 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9840000000000004e-06 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.867785+0000", 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5616000000000001e-05 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:21.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889845+0000", 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022060283 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889913+0000", 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8104999999999999e-05 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.889966+0000", 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2340999999999999e-05 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:21.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7283 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1703936~2106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.472384165000001, 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02059619, 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7283 2024-09-18T00:58:21.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473500+0000", 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:21.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473501+0000", 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473507+0000", 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2939999999999996e-06 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473522+0000", 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5386000000000001e-05 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473534+0000", 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1917e-05 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493796+0000", 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020262184999999999 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.493907+0000", 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011105800000000001 2024-09-18T00:58:22.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494097+0000", 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018939299999999999 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7284 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.472334226000001, 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020722913999999999, 2024-09-18T00:58:22.002 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7284 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473550+0000", 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473551+0000", 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999999e-07 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473556+0000", 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8949999999999997e-06 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473571+0000", 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.486e-05 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473603+0000", 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2375000000000001e-05 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494129+0000", 2024-09-18T00:58:22.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020525244000000002 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494206+0000", 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7601000000000005e-05 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494273+0000", 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7101999999999996e-05 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7292 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.472263955000003, 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020882831000000001, 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7292 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.007 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473621+0000", 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473622+0000", 2024-09-18T00:58:22.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.78e-07 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473626+0000", 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7330000000000003e-06 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473640+0000", 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4034e-05 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473651+0000", 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0937999999999999e-05 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494300+0000", 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020649238 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494392+0000", 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1184e-05 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494504+0000", 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111826 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7293 3.a 3:56466495:::rbd_data.117ce245106a.0000000000000007:5 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.472218638000001, 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021019589000000002, 2024-09-18T00:58:22.011 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7293 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473666+0000", 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473667+0000", 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6600000000000005e-07 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.473682+0000", 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.483e-05 2024-09-18T00:58:22.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.477101+0000", 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034192710000000002 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.477120+0000", 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9202999999999999e-05 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494533+0000", 2024-09-18T00:58:22.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.017412838999999999 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494603+0000", 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9947e-05 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.494686+0000", 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2633000000000007e-05 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15500 3.11 3:8ed6a97b:::rbd_data.117c53e529c1.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.131366532000001, 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046818956000000002, 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15500 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814521+0000", 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.977e-06 2024-09-18T00:58:22.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814536+0000", 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.487e-05 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814572+0000", 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5567e-05 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814588+0000", 2024-09-18T00:58:22.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6546e-05 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814675+0000", 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6675999999999995e-05 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814839+0000", 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016414199999999999 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:22.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.819908+0000", 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050695280000000002 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861139+0000", 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041230771999999999 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861208+0000", 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8722999999999999e-05 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861277+0000", 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9246999999999996e-05 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861337+0000", 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9908000000000003e-05 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:21989 3.1d 3:beac871e:::rbd_object_map.117c21a2e20a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.416499896000001, 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.021103017000000002, 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 21989 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529385+0000", 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2399999999999999e-07 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529389+0000", 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3759999999999999e-06 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529398+0000", 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4080000000000004e-06 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529404+0000", 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8919999999999997e-06 2024-09-18T00:58:22.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.544686+0000", 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015282662000000001 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.544799+0000", 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011215800000000001 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.545254+0000", 2024-09-18T00:58:22.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00045577600000000002 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.547172+0000", 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019178999999999999 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550317+0000", 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0031449310000000001 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:22.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550437+0000", 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119963 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.550488+0000", 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0526999999999999e-05 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:21990 3.1d 3:beac871e:::rbd_object_map.117c21a2e20a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.416471448999999, 2024-09-18T00:58:22.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023815820000000001, 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 21990 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529413+0000", 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529414+0000", 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529417+0000", 2024-09-18T00:58:22.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2849999999999999e-06 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529425+0000", 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7740000000000004e-06 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.529431+0000", 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8939999999999998e-06 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546090+0000", 2024-09-18T00:58:22.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.016659706 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546185+0000", 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4889999999999994e-05 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.546569+0000", 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038352599999999997 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:22.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.547692+0000", 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001123024 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553127+0000", 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0054353630000000003 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553188+0000", 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1247999999999994e-05 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:22.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:02.553229+0000", 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0646999999999999e-05 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26860 3.a 3:523ff797:::rbd_data.117c4c7db1a0.0000000000000039:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.573192412999999, 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028475341000000001, 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:22.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26860 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372693+0000", 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0550000000000001e-06 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372699+0000", 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372714+0000", 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4673e-05 2024-09-18T00:58:22.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372722+0000", 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4659999999999999e-06 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372787+0000", 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5288000000000002e-05 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372874+0000", 2024-09-18T00:58:22.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6485999999999998e-05 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.377992+0000", 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051177159999999996 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401008+0000", 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02301603 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:22.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401026+0000", 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712e-05 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401082+0000", 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5648e-05 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401168+0000", 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5476999999999995e-05 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.038 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:52308 3.d 3:b0cf5b15:::rbd_data.117cc4c72933.0000000000000015:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:21.432882+0000", 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.51300290999999998, 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.023271706999999999, 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:22.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 52308 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.432882+0000", 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.432882+0000", 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.432883+0000", 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7010000000000001e-06 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.432891+0000", 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.593e-06 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.432910+0000", 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8649000000000001e-05 2024-09-18T00:58:22.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.432918+0000", 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0560000000000005e-06 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.432974+0000", 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5846e-05 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.433025+0000", 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1338000000000002e-05 2024-09-18T00:58:22.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.435584+0000", 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025589749999999998 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.456064+0000", 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.020480485 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.456090+0000", 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5559e-05 2024-09-18T00:58:22.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.456123+0000", 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3034000000000003e-05 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.456153+0000", 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0471e-05 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.044 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:52309 3.d 3:b4db4490:::rbd_data.117cc4c72933.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:21.434106+0000", 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.51177815599999998, 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.022164289, 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 52309 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.434106+0000", 2024-09-18T00:58:22.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.434106+0000", 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.434109+0000", 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0700000000000001e-06 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:22.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.434120+0000", 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1467e-05 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.434150+0000", 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9853e-05 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.434172+0000", 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1756e-05 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:22.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.449028+0000", 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.014856482000000001 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.449081+0000", 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3266999999999997e-05 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.450081+0000", 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099998399999999999 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:22.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.456165+0000", 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060833390000000001 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.456199+0000", 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4173e-05 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.456232+0000", 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3386000000000002e-05 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:22.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:21.456271+0000", 2024-09-18T00:58:22.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8512e-05 2024-09-18T00:58:22.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.050 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:22.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:22.050 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:22.050 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:22.051 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:22.052 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:22.046+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:22.152 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:22.146+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:22.181 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:22.181 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:22.181 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:22.181 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:22.195 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:22.207 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T00:58:22.236 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T00:58:22.281 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:22.278+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:22.353 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:22.350+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:22.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:22.360 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:22.360 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:22.360 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:22.360 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:22.360 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:22.374 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:22.461 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:22.458+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:22.473 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-18T00:58:22.540 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:22.540 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:22.558 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:22.550+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:22.558 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1670 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1335296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.384267659000002, 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083399740999999999, 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1670 2024-09-18T00:58:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166573+0000", 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1829999999999999e-06 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166578+0000", 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0030000000000002e-06 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166594+0000", 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6470999999999999e-05 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166607+0000", 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2831e-05 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249706+0000", 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083099022999999994 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249843+0000", 2024-09-18T00:58:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136518 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249971+0000", 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012871200000000001 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1671 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1376256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.384216881999997, 2024-09-18T00:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083700574, 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1671 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166623+0000", 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.066e-06 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166641+0000", 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7728e-05 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166659+0000", 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7850999999999999e-05 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166672+0000", 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3020000000000001e-05 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250034+0000", 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083362085000000002 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250145+0000", 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111045 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250323+0000", 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017777900000000001 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1672 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.384143422999998, 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083853480999999994, 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1672 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166697+0000", 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0920000000000001e-06 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166702+0000", 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0799999999999996e-06 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166718+0000", 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6167999999999998e-05 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166745+0000", 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7177000000000001e-05 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250382+0000", 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083636792000000001 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250476+0000", 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250549+0000", 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3282999999999996e-05 2024-09-18T00:58:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1673 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.384076315000002, 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084151880999999998, 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1673 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166764+0000", 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166770+0000", 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5969999999999999e-06 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166786+0000", 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6439999999999998e-05 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166799+0000", 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2934e-05 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250605+0000", 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083806357999999997 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250724+0000", 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011821500000000001 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250915+0000", 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000191251 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1674 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.383976730000001, 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084239101999999996, 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1674 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166864+0000", 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3629999999999999e-06 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166883+0000", 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9083e-05 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166902+0000", 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8711000000000001e-05 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166915+0000", 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3375e-05 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250962+0000", 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084047254000000002 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251029+0000", 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6315000000000005e-05 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251102+0000", 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3001000000000001e-05 2024-09-18T00:58:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1675 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1540096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.383904878999999, 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084536891000000003, 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1675 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166936+0000", 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2720000000000001e-06 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166941+0000", 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7370000000000003e-06 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166958+0000", 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166972+0000", 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3569e-05 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251159+0000", 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084187467000000002 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251300+0000", 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014059200000000001 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251471+0000", 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017168000000000001 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1676 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.383851999000001, 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084738637000000006, 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1676 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166988+0000", 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166993+0000", 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8339999999999998e-06 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167009+0000", 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6028e-05 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167022+0000", 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2675e-05 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251517+0000", 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084494975999999999 2024-09-18T00:58:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251576+0000", 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8644999999999998e-05 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251726+0000", 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015039 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1677 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.383800328, 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085055403000000002, 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1677 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167040+0000", 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167046+0000", 2024-09-18T00:58:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2024-09-18T00:58:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167062+0000", 2024-09-18T00:58:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6354e-05 2024-09-18T00:58:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167075+0000", 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2965e-05 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251789+0000", 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084713940000000001 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251916+0000", 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000127176 2024-09-18T00:58:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252094+0000", 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017833599999999999 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1678 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.383748564999998, 2024-09-18T00:58:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085316746999999998, 2024-09-18T00:58:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1678 2024-09-18T00:58:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167092+0000", 2024-09-18T00:58:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:58:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167097+0000", 2024-09-18T00:58:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7029999999999997e-06 2024-09-18T00:58:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167113+0000", 2024-09-18T00:58:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-18T00:58:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167125+0000", 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2517000000000001e-05 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252157+0000", 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085031933000000004 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252277+0000", 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119708 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252407+0000", 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013031800000000001 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1679 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.383697845999997, 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085557195000000003, 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1679 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167143+0000", 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167170+0000", 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7523000000000001e-05 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167189+0000", 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8530999999999999e-05 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167215+0000", 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6676999999999999e-05 2024-09-18T00:58:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252445+0000", 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085229416000000002 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252529+0000", 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4740000000000005e-05 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252699+0000", 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016920800000000001 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1680 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.383604388999998, 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085810110999999994, 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1680 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167236+0000", 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.048e-06 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167242+0000", 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6459999999999998e-06 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167259+0000", 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.696e-05 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167271+0000", 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.292e-05 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252839+0000", 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085567292000000003 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252924+0000", 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5511000000000002e-05 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253045+0000", 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120734 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1681 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.383498293000002, 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085887343000000005, 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1681 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167342+0000", 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167348+0000", 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4519999999999996e-06 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167372+0000", 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4261999999999999e-05 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167400+0000", 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253091+0000", 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085690895000000003 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253154+0000", 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3262000000000005e-05 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253228+0000", 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4318000000000002e-05 2024-09-18T00:58:22.612 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1682 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.383420213000001, 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.086061112999999995, 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.613 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1682 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168114+0000", 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00069532600000000002 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168123+0000", 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8319999999999995e-06 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168147+0000", 2024-09-18T00:58:22.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3536999999999998e-05 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168163+0000", 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6317e-05 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253274+0000", 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085111390999999995 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253362+0000", 2024-09-18T00:58:22.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7384000000000002e-05 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253480+0000", 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011832600000000001 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1683 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1867776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.382643082000001, 2024-09-18T00:58:22.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085456638000000001, 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1683 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169938+0000", 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017419079999999999 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169951+0000", 2024-09-18T00:58:22.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2534999999999999e-05 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169990+0000", 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9004999999999997e-05 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170011+0000", 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.156e-05 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253534+0000", 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083522759000000002 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253585+0000", 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1316999999999997e-05 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253653+0000", 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7553999999999997e-05 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.621 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1684 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.380793445999998, 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083787188999999998, 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1684 2024-09-18T00:58:22.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170048+0000", 2024-09-18T00:58:22.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.674e-06 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170055+0000", 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3549999999999999e-06 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170073+0000", 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7662000000000001e-05 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170087+0000", 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4229e-05 2024-09-18T00:58:22.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253701+0000", 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083614346000000006 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253752+0000", 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0760000000000002e-05 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253833+0000", 2024-09-18T00:58:22.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1162999999999995e-05 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1685 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.380732649000002, 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083900785000000005, 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.626 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1685 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170108+0000", 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.229e-06 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170114+0000", 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0079999999999999e-06 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170132+0000", 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7734e-05 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170145+0000", 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2977e-05 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253866+0000", 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083721208000000005 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253934+0000", 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.834e-05 2024-09-18T00:58:22.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254007+0000", 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3288999999999996e-05 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1686 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.380677437999999, 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084083526000000006, 2024-09-18T00:58:22.630 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1686 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170163+0000", 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1909999999999999e-06 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170180+0000", 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7323e-05 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170198+0000", 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7723e-05 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170212+0000", 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3645999999999999e-05 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254050+0000", 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083838115000000005 2024-09-18T00:58:22.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254136+0000", 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6106999999999994e-05 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254245+0000", 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010942099999999999 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1687 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.634 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.380609135999997, 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084186574, 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1687 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:22.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170647+0000", 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416649 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170655+0000", 2024-09-18T00:58:22.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6510000000000007e-06 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170683+0000", 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7093999999999998e-05 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170701+0000", 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8383e-05 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254283+0000", 2024-09-18T00:58:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083582091999999997 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254339+0000", 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6187999999999998e-05 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254417+0000", 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7516999999999997e-05 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1688 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.380113422000001, 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083931026000000006, 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1688 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173553+0000", 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002827333 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173566+0000", 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3276e-05 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173603+0000", 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6177000000000003e-05 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173628+0000", 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5343000000000001e-05 2024-09-18T00:58:22.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254460+0000", 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.080831745999999996 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254547+0000", 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7045000000000004e-05 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254657+0000", 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011010600000000001 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.775976219999997, 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:22.643 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:22.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:22.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:58:22.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:58:22.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:22.648 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:22.649 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:22.649 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:22.650 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:22.650 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:22.650 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:22.651 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:22.651 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:22.651 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:22.652 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:22.655 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:22.650+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:22.661 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:22.661 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:22.682 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:22.703 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:22.703 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:22.719 INFO:teuthology.orchestra.run.smithi042.stdout:2063 read 0xba8873e thru 0xba939d4 (0xb297 bytes) 2024-09-18T00:58:22.756 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:22.750+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:22.762 INFO:teuthology.orchestra.run.smithi042.stdout:2065 write 0xdb7e1b5 thru 0xdb810b2 (0x2efe bytes) 2024-09-18T00:58:22.766 INFO:teuthology.orchestra.run.smithi042.stdout:2066 read 0xdb6ef1c thru 0xdb7e3bb (0xf4a0 bytes) 2024-09-18T00:58:22.766 INFO:teuthology.orchestra.run.smithi042.stdout:2067 write 0x5a8d68d thru 0x5a99c15 (0xc589 bytes) 2024-09-18T00:58:22.803 INFO:teuthology.orchestra.run.smithi042.stdout:2068 read 0xa853410 thru 0xa85beeb (0x8adc bytes) 2024-09-18T00:58:22.824 INFO:teuthology.orchestra.run.smithi042.stdout:2069 trunc from 0xdb810b3 to 0x484690a 2024-09-18T00:58:22.852 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:22.853 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:22.857 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:22.854+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:22.874 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:22.911 INFO:teuthology.orchestra.run.smithi042.stdout:2070 read 0x3ad624a thru 0x3adf5c2 (0x9379 bytes) 2024-09-18T00:58:22.911 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:22.911 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:22.912 INFO:teuthology.orchestra.run.smithi042.stdout:2073 read 0x3dde52e thru 0x3de247c (0x3f4f bytes) 2024-09-18T00:58:22.926 INFO:teuthology.orchestra.run.smithi042.stdout:2074 write 0xb076ebc thru 0xb07bb6d (0x4cb2 bytes) 2024-09-18T00:58:22.931 INFO:teuthology.orchestra.run.smithi042.stdout:2075 punch from 0x603a6eb to 0x603e9b5, (0x42ca bytes) 2024-09-18T00:58:22.940 INFO:teuthology.orchestra.run.smithi042.stdout:2076 write 0xcda1b57 thru 0xcdad93e (0xbde8 bytes) 2024-09-18T00:58:22.945 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:2078 write 0xe5d1d9e thru 0xe5d9b90 (0x7df3 bytes) 2024-09-18T00:58:22.958 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:22.954+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:22.959 INFO:teuthology.orchestra.run.smithi042.stdout:2080 read 0xe60517 thru 0xe628c5 (0x23af bytes) 2024-09-18T00:58:22.965 INFO:teuthology.orchestra.run.smithi042.stdout:2081 trunc from 0xe5d9b91 to 0xc199017 2024-09-18T00:58:23.012 INFO:teuthology.orchestra.run.smithi042.stdout:2082 trunc from 0xc199017 to 0x68a92f7 2024-09-18T00:58:23.028 INFO:teuthology.orchestra.run.smithi042.stdout:2083 write 0x453797f thru 0x453b3c6 (0x3a48 bytes) 2024-09-18T00:58:23.031 INFO:teuthology.orchestra.run.smithi042.stdout:2084 read 0x26f410 thru 0x2758e3 (0x64d4 bytes) 2024-09-18T00:58:23.059 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:23.059 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:23.060 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:23.054+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:23.076 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:23.085 INFO:teuthology.orchestra.run.smithi042.stdout:2087 write 0xea61680 thru 0xea6985b (0x81dc bytes) 2024-09-18T00:58:23.092 INFO:teuthology.orchestra.run.smithi042.stdout:2088 read 0x585b61d thru 0x586b4c2 (0xfea6 bytes) 2024-09-18T00:58:23.092 INFO:teuthology.orchestra.run.smithi042.stdout:2090 read 0xa6ab8a4 thru 0xa6b279e (0x6efb bytes) 2024-09-18T00:58:23.092 INFO:teuthology.orchestra.run.smithi042.stdout:2091 punch from 0x6d8b8d3 to 0x6d93358, (0x7a85 bytes) 2024-09-18T00:58:23.092 INFO:teuthology.orchestra.run.smithi042.stdout:2092 read 0x6963b07 thru 0x697364b (0xfb45 bytes) 2024-09-18T00:58:23.092 INFO:teuthology.orchestra.run.smithi042.stdout:2094 read 0x6e3a2d1 thru 0x6e46cdf (0xca0f bytes) 2024-09-18T00:58:23.092 INFO:teuthology.orchestra.run.smithi042.stdout:2095 write 0xce5c965 thru 0xce5d7d0 (0xe6c bytes) 2024-09-18T00:58:23.095 INFO:teuthology.orchestra.run.smithi042.stdout:2096 punch from 0x4ae64f3 to 0x4ae98b0, (0x33bd bytes) 2024-09-18T00:58:23.096 INFO:teuthology.orchestra.run.smithi042.stdout:2097 punch from 0xaa0ec7a to 0xaa1a5d2, (0xb958 bytes) 2024-09-18T00:58:23.096 INFO:teuthology.orchestra.run.smithi042.stdout:2098 writesame 0x48b3e1e thru 0x48b519c data_size 0x17(0x137f bytes) 2024-09-18T00:58:23.098 INFO:teuthology.orchestra.run.smithi042.stdout:2099 writesame 0xc3668f6 thru 0xc367094 data_size 0x1(0x79f bytes) 2024-09-18T00:58:23.104 INFO:teuthology.orchestra.run.smithi042.stdout:2101 trunc from 0xea6985c to 0xdaa20c6 2024-09-18T00:58:23.120 INFO:teuthology.orchestra.run.smithi042.stdout:2103 write 0xb65c0e8 thru 0xb668777 (0xc690 bytes) 2024-09-18T00:58:23.126 INFO:teuthology.orchestra.run.smithi042.stdout:2104 write 0xcbe2e2b thru 0xcbe6e06 (0x3fdc bytes) 2024-09-18T00:58:23.131 INFO:teuthology.orchestra.run.smithi042.stdout:2105 write 0x8f010e5 thru 0x8f07409 (0x6325 bytes) 2024-09-18T00:58:23.137 INFO:teuthology.orchestra.run.smithi042.stdout:2106 writesame 0xd4f8867 thru 0xd4fe443 data_size 0x43(0x5bdd bytes) 2024-09-18T00:58:23.144 INFO:teuthology.orchestra.run.smithi042.stdout:2110 punch from 0xd93ba52 to 0xd943a7a, (0x8028 bytes) 2024-09-18T00:58:23.144 INFO:teuthology.orchestra.run.smithi042.stdout:2111 write 0x2e26288 thru 0x2e307e2 (0xa55b bytes) 2024-09-18T00:58:23.165 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:23.162+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:23.203 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:23.203 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:23.203 INFO:teuthology.orchestra.run.smithi042.stdout:2116 punch from 0xc1d4177 to 0xc1d7511, (0x339a bytes) 2024-09-18T00:58:23.203 INFO:teuthology.orchestra.run.smithi042.stdout:2118 clone 21 order 21 su 16384 sc 11 2024-09-18T00:58:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T00:58:23.226 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:58:23.227 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T00:58:23.252 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:23.252 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:23.261 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:23.258+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:23.266 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:23.362 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:23.358+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:23.434 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:23.434 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:23.448 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:23.489 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:23.486+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:23.563 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:23.558+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:23.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-18T00:58:23.598+0000 7f3f75753640 0 --2- 172.21.15.42:0/2330303638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f700730a0 0x7f3f701d80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-18T00:58:23.613 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:23.613 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:23.627 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:23.664 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:23.658+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:23.765 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:23.758+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:23.792 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:23.792 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:23.809 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:23.866 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:23.862+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:23.928 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:23.928 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:23.942 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:23.985 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:23.982+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:24.068 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:24.062+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:24.103 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:24.103 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:24.117 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:24.205 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:24.201+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T00:58:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:58:24.258 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T00:58:24.269 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:24.265+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:24.283 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:24.283 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:24.298 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:24.315 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone20 from 0xdaa20c6 (overlap 0x484690a) to 0x15aec4f 2024-09-18T00:58:24.373 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:24.366+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:24.473 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:24.466+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:24.485 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:24.485 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:24.504 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:24.527 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-18T00:58:24.574 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:24.570+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:24.647 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:24.647 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:24.662 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:24.710 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:24.706+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:24.725 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:24.776 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:24.773+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:24.824 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:24.824 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:24.838 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:24.862 INFO:teuthology.orchestra.run.smithi042.stdout:2119 write 0x8bc72f2 thru 0x8bce811 (0x7520 bytes) 2024-09-18T00:58:24.877 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:24.869+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:24.888 INFO:teuthology.orchestra.run.smithi042.stdout:2120 punch from 0xa9babd3 to 0xa9c7369, (0xc796 bytes) 2024-09-18T00:58:24.900 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:24.900 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:24.900 INFO:teuthology.orchestra.run.smithi042.stdout:2121 trunc from 0xdaa20c6 to 0x342a56c 2024-09-18T00:58:24.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:24.929 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:24.961 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:24.962 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:24.977 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:24.970+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:24.989 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:24.989 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:24.998 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:61341 3.1f e62/24 3:f8297710:::rbd_data.117c1b4dcc93.000000000000001a:head v 62'184, mlcod=60'181)", 2024-09-18T00:58:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:24.996860+0000", 2024-09-18T00:58:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.00050154099999999999, 2024-09-18T00:58:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052762100000000004, 2024-09-18T00:58:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:24.996860+0000", 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:24.996860+0000", 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:24.996862+0000", 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3960000000000001e-06 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:24.996870+0000", 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7199999999999995e-06 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:24.996893+0000", 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2634999999999999e-05 2024-09-18T00:58:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:24.996903+0000", 2024-09-18T00:58:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0278999999999999e-05 2024-09-18T00:58:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:24.996967+0000", 2024-09-18T00:58:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3979999999999994e-05 2024-09-18T00:58:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:24.996982+0000", 2024-09-18T00:58:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4557e-05 2024-09-18T00:58:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:58:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 1 2024-09-18T00:58:25.002 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:25.016 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:25.017 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:25.018 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:25.034 INFO:teuthology.orchestra.run.smithi042.stdout:2122 write 0x9191582 thru 0x919ebde (0xd65d bytes) 2024-09-18T00:58:25.036 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:25.036 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:25.043 INFO:teuthology.orchestra.run.smithi042.stdout:2123 trunc from 0x919ebdf to 0x8a2a1d2 2024-09-18T00:58:25.051 INFO:teuthology.orchestra.run.smithi042.stdout:2124 read 0x672d91c thru 0x672e197 (0x87c bytes) 2024-09-18T00:58:25.051 INFO:teuthology.orchestra.run.smithi042.stdout:2126 punch from 0x2264aa0 to 0x22679f0, (0x2f50 bytes) 2024-09-18T00:58:25.051 INFO:teuthology.orchestra.run.smithi042.stdout:2127 read 0x72069d0 thru 0x72095cd (0x2bfe bytes) 2024-09-18T00:58:25.051 INFO:teuthology.orchestra.run.smithi042.stdout:2131 read 0x6a38f6c thru 0x6a420a0 (0x9135 bytes) 2024-09-18T00:58:25.051 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:25.053 INFO:teuthology.orchestra.run.smithi042.stdout:2134 trunc from 0x8a2a1d2 to 0xa6faec0 2024-09-18T00:58:25.059 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:25.060 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:25.060 INFO:teuthology.orchestra.run.smithi042.stdout:2135 writesame 0x6da6b6b thru 0x6dae2fc data_size 0x5(0x7792 bytes) 2024-09-18T00:58:25.067 INFO:teuthology.orchestra.run.smithi042.stdout:2136 read 0xa902c2 thru 0xa92fc5 (0x2d04 bytes) 2024-09-18T00:58:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:2137 read 0x30a23c3 thru 0x30a554d (0x318b bytes) 2024-09-18T00:58:25.078 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:25.074+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:25.087 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:25.088 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:25.100 INFO:teuthology.orchestra.run.smithi042.stdout:2138 write 0xc355422 thru 0xc362ee5 (0xdac4 bytes) 2024-09-18T00:58:25.110 INFO:teuthology.orchestra.run.smithi042.stdout:2139 write 0xae4805b thru 0xae4b184 (0x312a bytes) 2024-09-18T00:58:25.118 INFO:teuthology.orchestra.run.smithi042.stdout:2141 trunc from 0xc362ee6 to 0xeaffaf2 2024-09-18T00:58:25.125 INFO:teuthology.orchestra.run.smithi042.stdout:2142 writesame 0xc62d9b0 thru 0xc63d42a data_size 0x1(0xfa7b bytes) 2024-09-18T00:58:25.157 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:25.157 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:25.157 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:25.159 INFO:teuthology.orchestra.run.smithi042.stdout:2144 read 0x191b690 thru 0x192a69a (0xf00b bytes) 2024-09-18T00:58:25.170 INFO:teuthology.orchestra.run.smithi042.stdout:2146 write 0x948816 thru 0x949b6e (0x1359 bytes) 2024-09-18T00:58:25.180 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:25.180 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:25.173+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:25.207 INFO:teuthology.orchestra.run.smithi042.stdout:2147 write 0xbe4abe9 thru 0xbe4ccdb (0x20f3 bytes) 2024-09-18T00:58:25.212 INFO:teuthology.orchestra.run.smithi042.stdout:2149 write 0x7abdd2a thru 0x7acdada (0xfdb1 bytes) 2024-09-18T00:58:25.224 INFO:teuthology.orchestra.run.smithi042.stdout:2150 write 0xc3ec7d8 thru 0xc3ef768 (0x2f91 bytes) 2024-09-18T00:58:25.224 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T00:58:25.234 INFO:teuthology.orchestra.run.smithi042.stdout:2151 trunc from 0xeaffaf2 to 0xbec1c1f 2024-09-18T00:58:25.249 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T00:58:25.249 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:58:25.255 INFO:teuthology.orchestra.run.smithi042.stdout:2153 read 0x9961a89 thru 0x9970c76 (0xf1ee bytes) 2024-09-18T00:58:25.255 INFO:teuthology.orchestra.run.smithi042.stdout:2155 punch from 0x559c734 to 0x55a7b67, (0xb433 bytes) 2024-09-18T00:58:25.258 INFO:teuthology.orchestra.run.smithi042.stdout:2158 trunc from 0xbec1c1f to 0x929f421 2024-09-18T00:58:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:2160 write 0x792f465 thru 0x792faf8 (0x694 bytes) 2024-09-18T00:58:25.280 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:25.274+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:2161 write 0x873cf26 thru 0x8745089 (0x8164 bytes) 2024-09-18T00:58:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:2164 writesame 0xcab5229 thru 0xcab5af6 data_size 0x2e(0x8ce bytes) 2024-09-18T00:58:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:2167 writesame 0xbf009fe thru 0xbf05216 data_size 0x1(0x4819 bytes) 2024-09-18T00:58:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:2168 write 0xdacc8c8 thru 0xdad9af0 (0xd229 bytes) 2024-09-18T00:58:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:2170 write 0x365ecc3 thru 0x36669bb (0x7cf9 bytes) 2024-09-18T00:58:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:2171 write 0xf84e6f thru 0xf94ac5 (0xfc57 bytes) 2024-09-18T00:58:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.049480037999999, 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:25.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:58:25.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7423 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.853181487999997, 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045296661000000002, 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7423 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485621+0000", 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485627+0000", 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.156e-06 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485633+0000", 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789999999999999e-06 2024-09-18T00:58:25.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530788+0000", 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045155001 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530853+0000", 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5634000000000002e-05 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530915+0000", 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2022999999999999e-05 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7424 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1146880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.853160498000001, 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045419355000000002, 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.353 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7424 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485642+0000", 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0640000000000001e-06 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485649+0000", 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485654+0000", 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2329999999999997e-06 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530941+0000", 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045286602000000002 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531004+0000", 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3455999999999997e-05 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531059+0000", 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4771000000000001e-05 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.853138985000001, 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:25.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:58:25.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:58:25.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.853117548, 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.362 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.853022221000003, 2024-09-18T00:58:25.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:58:25.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.852996781999998, 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.371 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:58:25.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.852974259, 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.375 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:58:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:58:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:58:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:58:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:58:25.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.852950819, 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:58:25.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:58:25.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.852927008999998, 2024-09-18T00:58:25.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:58:25.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.388 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.852904643999999, 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.389 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:58:25.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:58:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:58:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:58:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.852878027000003, 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:58:25.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:58:25.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:58:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:58:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.400 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.848726255999999, 2024-09-18T00:58:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:58:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.401 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:58:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:25.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:58:25.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:58:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:58:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:58:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:58:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:58:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:58:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.406 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.848705512999999, 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:58:25.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:58:25.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.848659947999998, 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:58:25.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:58:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:58:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:58:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:58:25.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:58:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:58:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:58:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:58:25.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:58:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.848633606999996, 2024-09-18T00:58:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:58:25.416 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:25.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.420 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.848610936999997, 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:58:25.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:25.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.848592564, 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:58:25.425 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:58:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:58:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:58:25.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.848572775999997, 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.430 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:58:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:58:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:58:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:58:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:58:25.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.848545905999998, 2024-09-18T00:58:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:58:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.435 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:58:25.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:58:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:58:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:58:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:58:25.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:58:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.440 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.440 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:25.442 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:25.443 INFO:teuthology.orchestra.run.smithi042.stdout:2172 read 0xabb455d thru 0xabb4e54 (0x8f8 bytes) 2024-09-18T00:58:25.443 INFO:teuthology.orchestra.run.smithi042.stdout:2173 write 0x6bec443 thru 0x6bf119c (0x4d5a bytes) 2024-09-18T00:58:25.443 INFO:teuthology.orchestra.run.smithi042.stdout:2175 write 0x1708147 thru 0x170906c (0xf26 bytes) 2024-09-18T00:58:25.443 INFO:teuthology.orchestra.run.smithi042.stdout:2176 read 0x1a8bfdd thru 0x1a94077 (0x809b bytes) 2024-09-18T00:58:25.443 INFO:teuthology.orchestra.run.smithi042.stdout:2180 read 0x203a48c thru 0x204503f (0xabb4 bytes) 2024-09-18T00:58:25.443 INFO:teuthology.orchestra.run.smithi042.stdout:2181 read 0x28b8e06 thru 0x28c68f4 (0xdaef bytes) 2024-09-18T00:58:25.443 INFO:teuthology.orchestra.run.smithi042.stdout:2183 write 0xd69faeb thru 0xd6a3c36 (0x414c bytes) 2024-09-18T00:58:25.444 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:25.437+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:25.449 INFO:teuthology.orchestra.run.smithi042.stdout:2184 read 0x9ea4f41 thru 0x9ea69d6 (0x1a96 bytes) 2024-09-18T00:58:25.449 INFO:teuthology.orchestra.run.smithi042.stdout:2185 write 0xb324e29 thru 0xb32b095 (0x626d bytes) 2024-09-18T00:58:25.453 INFO:teuthology.orchestra.run.smithi042.stdout:2186 trunc from 0xdad9af1 to 0xe9ff346 2024-09-18T00:58:25.462 INFO:teuthology.orchestra.run.smithi042.stdout:2187 write 0xb2418df thru 0xb247b60 (0x6282 bytes) 2024-09-18T00:58:25.466 INFO:teuthology.orchestra.run.smithi042.stdout:2188 read 0xb558bf3 thru 0xb559b3a (0xf48 bytes) 2024-09-18T00:58:25.467 INFO:teuthology.orchestra.run.smithi042.stdout:2189 write 0x3835fd9 thru 0x3842135 (0xc15d bytes) 2024-09-18T00:58:25.469 INFO:teuthology.orchestra.run.smithi042.stdout:2190 write 0xd97d863 thru 0xd986170 (0x890e bytes) 2024-09-18T00:58:25.475 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:25.476 INFO:teuthology.orchestra.run.smithi042.stdout:2192 read 0x4626edd thru 0x462b403 (0x4527 bytes) 2024-09-18T00:58:25.476 INFO:teuthology.orchestra.run.smithi042.stdout:2195 punch from 0x4d0c34f to 0x4d1805c, (0xbd0d bytes) 2024-09-18T00:58:25.476 INFO:teuthology.orchestra.run.smithi042.stdout:2196 write 0x2cdee1d thru 0x2ceb289 (0xc46d bytes) 2024-09-18T00:58:25.477 INFO:teuthology.orchestra.run.smithi042.stdout:2197 write 0xa2a94a3 thru 0xa2b1608 (0x8166 bytes) 2024-09-18T00:58:25.483 INFO:teuthology.orchestra.run.smithi042.stdout:2198 read 0xcfb1f3e thru 0xcfb3877 (0x193a bytes) 2024-09-18T00:58:25.483 INFO:teuthology.orchestra.run.smithi042.stdout:2199 write 0x18f5484 thru 0x190081d (0xb39a bytes) 2024-09-18T00:58:25.487 INFO:teuthology.orchestra.run.smithi042.stdout:2200 read 0x4dc403d thru 0x4dc4bad (0xb71 bytes) 2024-09-18T00:58:25.487 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:25.487 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:25.487 INFO:teuthology.orchestra.run.smithi042.stdout:2203 read 0x48c754b thru 0x48c8156 (0xc0c bytes) 2024-09-18T00:58:25.487 INFO:teuthology.orchestra.run.smithi042.stdout:2207 write 0x93fb886 thru 0x93fea73 (0x31ee bytes) 2024-09-18T00:58:25.489 INFO:teuthology.orchestra.run.smithi042.stdout:2211 read 0xd63d542 thru 0xd646607 (0x90c6 bytes) 2024-09-18T00:58:25.489 INFO:teuthology.orchestra.run.smithi042.stdout:2212 punch from 0x9e66a07 to 0x9e6aa14, (0x400d bytes) 2024-09-18T00:58:25.501 INFO:teuthology.orchestra.run.smithi042.stdout:2213 trunc from 0xe9ff346 to 0x4369398 2024-09-18T00:58:25.522 INFO:teuthology.orchestra.run.smithi042.stdout:2215 read 0x17a13df thru 0x17ae3ba (0xcfdc bytes) 2024-09-18T00:58:25.523 INFO:teuthology.orchestra.run.smithi042.stdout:2216 read 0x428185d thru 0x428d86f (0xc013 bytes) 2024-09-18T00:58:25.523 INFO:teuthology.orchestra.run.smithi042.stdout:2217 write 0x411c4b1 thru 0x411e8fc (0x244c bytes) 2024-09-18T00:58:25.524 INFO:teuthology.orchestra.run.smithi042.stdout:2218 read 0x387ee77 thru 0x388b6a1 (0xc82b bytes) 2024-09-18T00:58:25.532 INFO:teuthology.orchestra.run.smithi042.stdout:2219 trunc from 0x4369398 to 0x9cb37c8 2024-09-18T00:58:25.542 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:25.538+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:25.542 INFO:teuthology.orchestra.run.smithi042.stdout:2220 trunc from 0x9cb37c8 to 0xdeff849 2024-09-18T00:58:25.549 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:25.564 INFO:teuthology.orchestra.run.smithi042.stdout:2222 trunc from 0xdeff849 to 0xf9a316 2024-09-18T00:58:25.583 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:63791 3.16 3:6a77db44:::rbd_data.117c1b4dcc93.0000000000000000:head [stat out=16b,truncate 1490944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.581812+0000", 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0012180660000000001, 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001245206, 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "waiting for sub ops", 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 63791 2024-09-18T00:58:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.581812+0000", 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.581812+0000", 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.581813+0000", 2024-09-18T00:58:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0160000000000001e-06 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.581818+0000", 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0329999999999999e-06 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.581834+0000", 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6050999999999999e-05 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.581842+0000", 2024-09-18T00:58:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0649999999999999e-06 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.581901+0000", 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9023000000000001e-05 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.581992+0000", 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1096999999999999e-05 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.582124+0000", 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000131774 2024-09-18T00:58:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.583047+0000", 2024-09-18T00:58:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00092321899999999997 2024-09-18T00:58:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:58:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 1 2024-09-18T00:58:25.588 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:25.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:25.621 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:58:25.622 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:58:25.644 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:25.637+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:25.645 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-09-18T00:58:25.222255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":524550827,"num_objects":1259,"num_object_clones":242,"num_object_copies":2518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1259,"num_whiteouts":620,"num_read":68177,"num_read_kb":332755,"num_write":7609,"num_write_kb":327184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6602,"ondisk_log_size":6602,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":619080,"kb_used_data":303252,"kb_used_omap":12,"kb_used_meta":307571,"kb_avail":754355640,"statfs":{"total":773094113280,"available":772460175360,"internally_reserved":0,"allocated":310530048,"data_stored":330457624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":314953264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":1472,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027080"},"pg_stats":[{"pgid":"3.1f","version":"56'174","reported_seq":1255,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.200336+0000","last_change":"2024-09-18T00:57:38.909849+0000","last_active":"2024-09-18T00:58:20.200336+0000","last_peered":"2024-09-18T00:58:20.200336+0000","last_clean":"2024-09-18T00:58:20.200336+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T00:58:20.200336+0000","last_undegraded":"2024-09-18T00:58:20.200336+0000","last_fullsized":"2024-09-18T00:58:20.200336+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:27:45.244510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001565786,"stat_sum":{"num_bytes":5807801,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":19,"num_read":2323,"num_read_kb":5553,"num_write":151,"num_write_kb":2637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"58'149","reported_seq":1227,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.659838+0000","last_change":"2024-09-18T00:57:38.908434+0000","last_active":"2024-09-18T00:58:20.659838+0000","last_peered":"2024-09-18T00:58:20.659838+0000","last_clean":"2024-09-18T00:58:20.659838+0000","last_became_active":"2024-09-18T00:57:36.897948+0000","last_became_peered":"2024-09-18T00:57:36.897948+0000","last_unstale":"2024-09-18T00:58:20.659838+0000","last_undegraded":"2024-09-18T00:58:20.659838+0000","last_fullsized":"2024-09-18T00:58:20.659838+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:44.221240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040574399999999998,"stat_sum":{"num_bytes":9427091,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":24,"num_read":1195,"num_read_kb":5732,"num_write":140,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"56'387","reported_seq":1238,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190770+0000","last_change":"2024-09-18T00:57:38.902294+0000","last_active":"2024-09-18T00:58:20.190770+0000","last_peered":"2024-09-18T00:58:20.190770+0000","last_clean":"2024-09-18T00:58:20.190770+0000","last_became_active":"2024-09-18T00:57:36.906790+0000","last_became_peered":"2024-09-18T00:57:36.906790+0000","last_unstale":"2024-09-18T00:58:20.190770+0000","last_undegraded":"2024-09-18T00:58:20.190770+0000","last_fullsized":"2024-09-18T00:58:20.190770+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:58:37.920323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039089,"stat_sum":{"num_bytes":10473306,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":24,"num_read":2271,"num_read_kb":12736,"num_write":591,"num_write_kb":2130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"56'314","reported_seq":1567,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.186931+0000","last_change":"2024-09-18T00:57:38.901600+0000","last_active":"2024-09-18T00:58:20.186931+0000","last_peered":"2024-09-18T00:58:20.186931+0000","last_clean":"2024-09-18T00:58:20.186931+0000","last_became_active":"2024-09-18T00:57:36.906003+0000","last_became_peered":"2024-09-18T00:57:36.906003+0000","last_unstale":"2024-09-18T00:58:20.186931+0000","last_undegraded":"2024-09-18T00:58:20.186931+0000","last_fullsized":"2024-09-18T00:58:20.186931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:39:14.537556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000711266,"stat_sum":{"num_bytes":4672646,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":2555,"num_read_kb":8820,"num_write":370,"num_write_kb":1830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"58'342","reported_seq":5404,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.661429+0000","last_change":"2024-09-18T00:57:38.921854+0000","last_active":"2024-09-18T00:58:20.661429+0000","last_peered":"2024-09-18T00:58:20.661429+0000","last_clean":"2024-09-18T00:58:20.661429+0000","last_became_active":"2024-09-18T00:57:36.913393+0000","last_became_peered":"2024-09-18T00:57:36.913393+0000","last_unstale":"2024-09-18T00:58:20.661429+0000","last_undegraded":"2024-09-18T00:58:20.661429+0000","last_fullsized":"2024-09-18T00:58:20.661429+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:35:13.924235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002554022,"stat_sum":{"num_bytes":28064553,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":6009,"num_read_kb":16867,"num_write":481,"num_write_kb":11875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"58'168","reported_seq":1695,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.822413+0000","last_change":"2024-09-18T00:57:38.901805+0000","last_active":"2024-09-18T00:58:20.822413+0000","last_peered":"2024-09-18T00:58:20.822413+0000","last_clean":"2024-09-18T00:58:20.822413+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T00:58:20.822413+0000","last_undegraded":"2024-09-18T00:58:20.822413+0000","last_fullsized":"2024-09-18T00:58:20.822413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:13.529614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098070899999999996,"stat_sum":{"num_bytes":26466557,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":17,"num_read":2384,"num_read_kb":8111,"num_write":136,"num_write_kb":764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"58'160","reported_seq":8632,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.879595+0000","last_change":"2024-09-18T00:57:38.922010+0000","last_active":"2024-09-18T00:58:20.879595+0000","last_peered":"2024-09-18T00:58:20.879595+0000","last_clean":"2024-09-18T00:58:20.879595+0000","last_became_active":"2024-09-18T00:57:36.910457+0000","last_became_peered":"2024-09-18T00:57:36.910457+0000","last_unstale":"2024-09-18T00:58:20.879595+0000","last_undegraded":"2024-09-18T00:58:20.879595+0000","last_fullsized":"2024-09-18T00:58:20.879595+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:46:12.174403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026842849999999998,"stat_sum":{"num_bytes":34158112,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":8465,"num_read_kb":26532,"num_write":154,"num_write_kb":13870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"58'370","reported_seq":1321,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.907359+0000","last_change":"2024-09-18T00:57:38.908144+0000","last_active":"2024-09-18T00:58:20.907359+0000","last_peered":"2024-09-18T00:58:20.907359+0000","last_clean":"2024-09-18T00:58:20.907359+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T00:58:20.907359+0000","last_undegraded":"2024-09-18T00:58:20.907359+0000","last_fullsized":"2024-09-18T00:58:20.907359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:04:12.939208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000871597,"stat_sum":{"num_bytes":29218606,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":2455,"num_read_kb":3397,"num_write":504,"num_write_kb":34655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"58'159","reported_seq":841,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.833465+0000","last_change":"2024-09-18T00:57:38.910038+0000","last_active":"2024-09-18T00:58:20.833465+0000","last_peered":"2024-09-18T00:58:20.833465+0000","last_clean":"2024-09-18T00:58:20.833465+0000","last_became_active":"2024-09-18T00:57:36.895915+0000","last_became_peered":"2024-09-18T00:57:36.895915+0000","last_unstale":"2024-09-18T00:58:20.833465+0000","last_undegraded":"2024-09-18T00:58:20.833465+0000","last_fullsized":"2024-09-18T00:58:20.833465+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:25.225803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001269769,"stat_sum":{"num_bytes":13162855,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":999,"num_read_kb":12929,"num_write":150,"num_write_kb":12595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"58'212","reported_seq":2845,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.649324+0000","last_change":"2024-09-18T00:57:39.381845+0000","last_active":"2024-09-18T00:58:20.649324+0000","last_peered":"2024-09-18T00:58:20.649324+0000","last_clean":"2024-09-18T00:58:20.649324+0000","last_became_active":"2024-09-18T00:57:36.894538+0000","last_became_peered":"2024-09-18T00:57:36.894538+0000","last_unstale":"2024-09-18T00:58:20.649324+0000","last_undegraded":"2024-09-18T00:58:20.649324+0000","last_fullsized":"2024-09-18T00:58:20.649324+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:58.064925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076552,"stat_sum":{"num_bytes":12181857,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":3140,"num_read_kb":14578,"num_write":335,"num_write_kb":16391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"56'123","reported_seq":754,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.216036+0000","last_change":"2024-09-18T00:57:38.909647+0000","last_active":"2024-09-18T00:58:20.216036+0000","last_peered":"2024-09-18T00:58:20.216036+0000","last_clean":"2024-09-18T00:58:20.216036+0000","last_became_active":"2024-09-18T00:57:36.902977+0000","last_became_peered":"2024-09-18T00:57:36.902977+0000","last_unstale":"2024-09-18T00:58:20.216036+0000","last_undegraded":"2024-09-18T00:58:20.216036+0000","last_fullsized":"2024-09-18T00:58:20.216036+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:56:23.484909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051351900000000004,"stat_sum":{"num_bytes":23986238,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":571,"num_read_kb":7729,"num_write":146,"num_write_kb":38402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"58'88","reported_seq":507,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.649599+0000","last_change":"2024-09-18T00:57:38.902578+0000","last_active":"2024-09-18T00:58:20.649599+0000","last_peered":"2024-09-18T00:58:20.649599+0000","last_clean":"2024-09-18T00:58:20.649599+0000","last_became_active":"2024-09-18T00:57:36.894977+0000","last_became_peered":"2024-09-18T00:57:36.894977+0000","last_unstale":"2024-09-18T00:58:20.649599+0000","last_undegraded":"2024-09-18T00:58:20.649599+0000","last_fullsized":"2024-09-18T00:58:20.649599+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:35:55.273404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028223319999999999,"stat_sum":{"num_bytes":5890208,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":357,"num_read_kb":762,"num_write":69,"num_write_kb":846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.191066+0000","last_change":"2024-09-18T00:57:29.328163+0000","last_active":"2024-09-18T00:58:20.191066+0000","last_peered":"2024-09-18T00:58:20.191066+0000","last_clean":"2024-09-18T00:58:20.191066+0000","last_became_active":"2024-09-18T00:57:27.311231+0000","last_became_peered":"2024-09-18T00:57:27.311231+0000","last_unstale":"2024-09-18T00:58:20.191066+0000","last_undegraded":"2024-09-18T00:58:20.191066+0000","last_fullsized":"2024-09-18T00:58:20.191066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:57.340012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020990940000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"56'195","reported_seq":1350,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.209670+0000","last_change":"2024-09-18T00:57:38.920098+0000","last_active":"2024-09-18T00:58:20.209670+0000","last_peered":"2024-09-18T00:58:20.209670+0000","last_clean":"2024-09-18T00:58:20.209670+0000","last_became_active":"2024-09-18T00:57:36.920599+0000","last_became_peered":"2024-09-18T00:57:36.920599+0000","last_unstale":"2024-09-18T00:58:20.209670+0000","last_undegraded":"2024-09-18T00:58:20.209670+0000","last_fullsized":"2024-09-18T00:58:20.209670+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:25:08.740023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276634,"stat_sum":{"num_bytes":10760753,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":21,"num_read":1685,"num_read_kb":14337,"num_write":262,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.209497+0000","last_change":"2024-09-18T00:57:29.327841+0000","last_active":"2024-09-18T00:58:20.209497+0000","last_peered":"2024-09-18T00:58:20.209497+0000","last_clean":"2024-09-18T00:58:20.209497+0000","last_became_active":"2024-09-18T00:57:27.315417+0000","last_became_peered":"2024-09-18T00:57:27.315417+0000","last_unstale":"2024-09-18T00:58:20.209497+0000","last_undegraded":"2024-09-18T00:58:20.209497+0000","last_fullsized":"2024-09-18T00:58:20.209497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:54.335334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"58'177","reported_seq":3690,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.652702+0000","last_change":"2024-09-18T00:57:38.920634+0000","last_active":"2024-09-18T00:58:20.652702+0000","last_peered":"2024-09-18T00:58:20.652702+0000","last_clean":"2024-09-18T00:58:20.652702+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T00:58:20.652702+0000","last_undegraded":"2024-09-18T00:58:20.652702+0000","last_fullsized":"2024-09-18T00:58:20.652702+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:20.716204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728301,"stat_sum":{"num_bytes":15265512,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":4180,"num_read_kb":9337,"num_write":144,"num_write_kb":6194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190971+0000","last_change":"2024-09-18T00:57:29.327995+0000","last_active":"2024-09-18T00:58:20.190971+0000","last_peered":"2024-09-18T00:58:20.190971+0000","last_clean":"2024-09-18T00:58:20.190971+0000","last_became_active":"2024-09-18T00:57:27.316238+0000","last_became_peered":"2024-09-18T00:57:27.316238+0000","last_unstale":"2024-09-18T00:58:20.190971+0000","last_undegraded":"2024-09-18T00:58:20.190971+0000","last_fullsized":"2024-09-18T00:58:20.190971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019441149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"58'123","reported_seq":711,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.875857+0000","last_change":"2024-09-18T00:57:39.056519+0000","last_active":"2024-09-18T00:58:20.875857+0000","last_peered":"2024-09-18T00:58:20.875857+0000","last_clean":"2024-09-18T00:58:20.875857+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T00:58:20.875857+0000","last_undegraded":"2024-09-18T00:58:20.875857+0000","last_fullsized":"2024-09-18T00:58:20.875857+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:01.547712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043106399999999999,"stat_sum":{"num_bytes":18056261,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":544,"num_read_kb":5637,"num_write":103,"num_write_kb":9328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.200354+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:58:20.200354+0000","last_peered":"2024-09-18T00:58:20.200354+0000","last_clean":"2024-09-18T00:58:20.200354+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:58:20.200354+0000","last_undegraded":"2024-09-18T00:58:20.200354+0000","last_fullsized":"2024-09-18T00:58:20.200354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"58'110","reported_seq":715,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.650546+0000","last_change":"2024-09-18T00:57:39.381374+0000","last_active":"2024-09-18T00:58:20.650546+0000","last_peered":"2024-09-18T00:58:20.650546+0000","last_clean":"2024-09-18T00:58:20.650546+0000","last_became_active":"2024-09-18T00:57:36.902781+0000","last_became_peered":"2024-09-18T00:57:36.902781+0000","last_unstale":"2024-09-18T00:58:20.650546+0000","last_undegraded":"2024-09-18T00:58:20.650546+0000","last_fullsized":"2024-09-18T00:58:20.650546+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:10:22.436994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000584944,"stat_sum":{"num_bytes":8305362,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":826,"num_read_kb":10701,"num_write":95,"num_write_kb":1315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"58'235","reported_seq":1205,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.334598+0000","last_change":"2024-09-18T00:57:38.909273+0000","last_active":"2024-09-18T00:58:20.334598+0000","last_peered":"2024-09-18T00:58:20.334598+0000","last_clean":"2024-09-18T00:58:20.334598+0000","last_became_active":"2024-09-18T00:57:36.906264+0000","last_became_peered":"2024-09-18T00:57:36.906264+0000","last_unstale":"2024-09-18T00:58:20.334598+0000","last_undegraded":"2024-09-18T00:58:20.334598+0000","last_fullsized":"2024-09-18T00:58:20.334598+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:29.318187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056105599999999999,"stat_sum":{"num_bytes":25332204,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":1676,"num_read_kb":2036,"num_write":336,"num_write_kb":15885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190674+0000","last_change":"2024-09-18T00:57:29.327436+0000","last_active":"2024-09-18T00:58:20.190674+0000","last_peered":"2024-09-18T00:58:20.190674+0000","last_clean":"2024-09-18T00:58:20.190674+0000","last_became_active":"2024-09-18T00:57:27.410600+0000","last_became_peered":"2024-09-18T00:57:27.410600+0000","last_unstale":"2024-09-18T00:58:20.190674+0000","last_undegraded":"2024-09-18T00:58:20.190674+0000","last_fullsized":"2024-09-18T00:58:20.190674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:59.380173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"58'224","reported_seq":4789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.285564+0000","last_change":"2024-09-18T00:57:38.907753+0000","last_active":"2024-09-18T00:58:20.285564+0000","last_peered":"2024-09-18T00:58:20.285564+0000","last_clean":"2024-09-18T00:58:20.285564+0000","last_became_active":"2024-09-18T00:57:36.907660+0000","last_became_peered":"2024-09-18T00:57:36.907660+0000","last_unstale":"2024-09-18T00:58:20.285564+0000","last_undegraded":"2024-09-18T00:58:20.285564+0000","last_fullsized":"2024-09-18T00:58:20.285564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:20:33.569644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090282999999999997,"stat_sum":{"num_bytes":14224233,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":25,"num_read":5032,"num_read_kb":13060,"num_write":275,"num_write_kb":3505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":93,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T00:58:22.286852+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:58:22.286852+0000","last_peered":"2024-09-18T00:58:22.286852+0000","last_clean":"2024-09-18T00:58:22.286852+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:58:22.286852+0000","last_undegraded":"2024-09-18T00:58:22.286852+0000","last_fullsized":"2024-09-18T00:58:22.286852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"58'173","reported_seq":1633,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.849849+0000","last_change":"2024-09-18T00:57:39.056604+0000","last_active":"2024-09-18T00:58:20.849849+0000","last_peered":"2024-09-18T00:58:20.849849+0000","last_clean":"2024-09-18T00:58:20.849849+0000","last_became_active":"2024-09-18T00:57:36.903081+0000","last_became_peered":"2024-09-18T00:57:36.903081+0000","last_unstale":"2024-09-18T00:58:20.849849+0000","last_undegraded":"2024-09-18T00:58:20.849849+0000","last_fullsized":"2024-09-18T00:58:20.849849+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:20.963144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045206800000000002,"stat_sum":{"num_bytes":10082907,"num_objects":45,"num_object_clones":9,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":2074,"num_read_kb":13164,"num_write":146,"num_write_kb":1232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190411+0000","last_change":"2024-09-18T00:57:29.327276+0000","last_active":"2024-09-18T00:58:20.190411+0000","last_peered":"2024-09-18T00:58:20.190411+0000","last_clean":"2024-09-18T00:58:20.190411+0000","last_became_active":"2024-09-18T00:57:27.316217+0000","last_became_peered":"2024-09-18T00:57:27.316217+0000","last_unstale":"2024-09-18T00:58:20.190411+0000","last_undegraded":"2024-09-18T00:58:20.190411+0000","last_fullsized":"2024-09-18T00:58:20.190411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:38.026720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012774469999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"56'125","reported_seq":521,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.209289+0000","last_change":"2024-09-18T00:57:38.919998+0000","last_active":"2024-09-18T00:58:20.209289+0000","last_peered":"2024-09-18T00:58:20.209289+0000","last_clean":"2024-09-18T00:58:20.209289+0000","last_became_active":"2024-09-18T00:57:36.906680+0000","last_became_peered":"2024-09-18T00:57:36.906680+0000","last_unstale":"2024-09-18T00:58:20.209289+0000","last_undegraded":"2024-09-18T00:58:20.209289+0000","last_fullsized":"2024-09-18T00:58:20.209289+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:16.072424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064609,"stat_sum":{"num_bytes":14095436,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":24,"num_read":360,"num_read_kb":2311,"num_write":103,"num_write_kb":33816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190856+0000","last_change":"2024-09-18T00:57:29.328733+0000","last_active":"2024-09-18T00:58:20.190856+0000","last_peered":"2024-09-18T00:58:20.190856+0000","last_clean":"2024-09-18T00:58:20.190856+0000","last_became_active":"2024-09-18T00:57:27.410396+0000","last_became_peered":"2024-09-18T00:57:27.410396+0000","last_unstale":"2024-09-18T00:58:20.190856+0000","last_undegraded":"2024-09-18T00:58:20.190856+0000","last_fullsized":"2024-09-18T00:58:20.190856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:27.438117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":240,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.196852+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:58:20.196852+0000","last_peered":"2024-09-18T00:58:20.196852+0000","last_clean":"2024-09-18T00:58:20.196852+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:58:20.196852+0000","last_undegraded":"2024-09-18T00:58:20.196852+0000","last_fullsized":"2024-09-18T00:58:20.196852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"58'149","reported_seq":1084,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.913643+0000","last_change":"2024-09-18T00:57:38.919534+0000","last_active":"2024-09-18T00:58:20.913643+0000","last_peered":"2024-09-18T00:58:20.913643+0000","last_clean":"2024-09-18T00:58:20.913643+0000","last_became_active":"2024-09-18T00:57:36.908955+0000","last_became_peered":"2024-09-18T00:57:36.908955+0000","last_unstale":"2024-09-18T00:58:20.913643+0000","last_undegraded":"2024-09-18T00:58:20.913643+0000","last_fullsized":"2024-09-18T00:58:20.913643+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:40.118305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077760500000000001,"stat_sum":{"num_bytes":31528837,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":972,"num_read_kb":14955,"num_write":155,"num_write_kb":26152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"58'121","reported_seq":780,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.902028+0000","last_change":"2024-09-18T00:57:38.899513+0000","last_active":"2024-09-18T00:58:20.902028+0000","last_peered":"2024-09-18T00:58:20.902028+0000","last_clean":"2024-09-18T00:58:20.902028+0000","last_became_active":"2024-09-18T00:57:36.911337+0000","last_became_peered":"2024-09-18T00:57:36.911337+0000","last_unstale":"2024-09-18T00:58:20.902028+0000","last_undegraded":"2024-09-18T00:58:20.902028+0000","last_fullsized":"2024-09-18T00:58:20.902028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:22:40.060688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061570699999999995,"stat_sum":{"num_bytes":15290136,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":18,"num_read":600,"num_read_kb":3357,"num_write":105,"num_write_kb":16064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"58'138","reported_seq":879,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.659321+0000","last_change":"2024-09-18T00:57:38.899904+0000","last_active":"2024-09-18T00:58:20.659321+0000","last_peered":"2024-09-18T00:58:20.659321+0000","last_clean":"2024-09-18T00:58:20.659321+0000","last_became_active":"2024-09-18T00:57:36.901837+0000","last_became_peered":"2024-09-18T00:57:36.901837+0000","last_unstale":"2024-09-18T00:58:20.659321+0000","last_undegraded":"2024-09-18T00:58:20.659321+0000","last_fullsized":"2024-09-18T00:58:20.659321+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:10.662378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097047300000000002,"stat_sum":{"num_bytes":11558357,"num_objects":43,"num_object_clones":5,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":718,"num_read_kb":12136,"num_write":113,"num_write_kb":8906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"60'122","reported_seq":1869,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T00:58:22.568467+0000","last_change":"2024-09-18T00:57:38.899763+0000","last_active":"2024-09-18T00:58:22.568467+0000","last_peered":"2024-09-18T00:58:22.568467+0000","last_clean":"2024-09-18T00:58:22.568467+0000","last_became_active":"2024-09-18T00:57:36.905869+0000","last_became_peered":"2024-09-18T00:57:36.905869+0000","last_unstale":"2024-09-18T00:58:22.568467+0000","last_undegraded":"2024-09-18T00:58:22.568467+0000","last_fullsized":"2024-09-18T00:58:22.568467+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:49.461946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097615600000000005,"stat_sum":{"num_bytes":12997052,"num_objects":51,"num_object_clones":15,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":19,"num_read":1888,"num_read_kb":24202,"num_write":84,"num_write_kb":2441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"58'266","reported_seq":2090,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.871483+0000","last_change":"2024-09-18T00:57:38.909165+0000","last_active":"2024-09-18T00:58:20.871483+0000","last_peered":"2024-09-18T00:58:20.871483+0000","last_clean":"2024-09-18T00:58:20.871483+0000","last_became_active":"2024-09-18T00:57:36.908288+0000","last_became_peered":"2024-09-18T00:57:36.908288+0000","last_unstale":"2024-09-18T00:58:20.871483+0000","last_undegraded":"2024-09-18T00:58:20.871483+0000","last_fullsized":"2024-09-18T00:58:20.871483+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:19:16.196155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044375399999999998,"stat_sum":{"num_bytes":16662412,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":2609,"num_read_kb":9294,"num_write":335,"num_write_kb":4553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"58'490","reported_seq":1492,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.883177+0000","last_change":"2024-09-18T00:57:38.903046+0000","last_active":"2024-09-18T00:58:20.883177+0000","last_peered":"2024-09-18T00:58:20.883177+0000","last_clean":"2024-09-18T00:58:20.883177+0000","last_became_active":"2024-09-18T00:57:36.900286+0000","last_became_peered":"2024-09-18T00:57:36.900286+0000","last_unstale":"2024-09-18T00:58:20.883177+0000","last_undegraded":"2024-09-18T00:58:20.883177+0000","last_fullsized":"2024-09-18T00:58:20.883177+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:33.280360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023201229999999999,"stat_sum":{"num_bytes":14935651,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":23,"num_read":3439,"num_read_kb":10352,"num_write":669,"num_write_kb":3385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"58'148","reported_seq":540,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.771776+0000","last_change":"2024-09-18T00:57:39.382152+0000","last_active":"2024-09-18T00:58:20.771776+0000","last_peered":"2024-09-18T00:58:20.771776+0000","last_clean":"2024-09-18T00:58:20.771776+0000","last_became_active":"2024-09-18T00:57:36.904791+0000","last_became_peered":"2024-09-18T00:57:36.904791+0000","last_unstale":"2024-09-18T00:58:20.771776+0000","last_undegraded":"2024-09-18T00:58:20.771776+0000","last_fullsized":"2024-09-18T00:58:20.771776+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:53:16.259577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090629000000000003,"stat_sum":{"num_bytes":8037756,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":700,"num_read_kb":2062,"num_write":124,"num_write_kb":762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"58'199","reported_seq":568,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.885793+0000","last_change":"2024-09-18T00:57:38.909597+0000","last_active":"2024-09-18T00:58:20.885793+0000","last_peered":"2024-09-18T00:58:20.885793+0000","last_clean":"2024-09-18T00:58:20.885793+0000","last_became_active":"2024-09-18T00:57:36.905462+0000","last_became_peered":"2024-09-18T00:57:36.905462+0000","last_unstale":"2024-09-18T00:58:20.885793+0000","last_undegraded":"2024-09-18T00:58:20.885793+0000","last_fullsized":"2024-09-18T00:58:20.885793+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:59:17.641292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081831799999999995,"stat_sum":{"num_bytes":23466562,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":820,"num_read_kb":6727,"num_write":248,"num_write_kb":6274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"58'191","reported_seq":2019,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.489364+0000","last_change":"2024-09-18T00:57:38.900906+0000","last_active":"2024-09-18T00:58:20.489364+0000","last_peered":"2024-09-18T00:58:20.489364+0000","last_clean":"2024-09-18T00:58:20.489364+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T00:58:20.489364+0000","last_undegraded":"2024-09-18T00:58:20.489364+0000","last_fullsized":"2024-09-18T00:58:20.489364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:14:20.481320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069386899999999995,"stat_sum":{"num_bytes":32329174,"num_objects":58,"num_object_clones":13,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":2404,"num_read_kb":34544,"num_write":161,"num_write_kb":24139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"58'112","reported_seq":628,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.832827+0000","last_change":"2024-09-18T00:57:38.899297+0000","last_active":"2024-09-18T00:58:20.832827+0000","last_peered":"2024-09-18T00:58:20.832827+0000","last_clean":"2024-09-18T00:58:20.832827+0000","last_became_active":"2024-09-18T00:57:36.897289+0000","last_became_peered":"2024-09-18T00:57:36.897289+0000","last_unstale":"2024-09-18T00:58:20.832827+0000","last_undegraded":"2024-09-18T00:58:20.832827+0000","last_fullsized":"2024-09-18T00:58:20.832827+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:24.696611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071487799999999995,"stat_sum":{"num_bytes":20350688,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":456,"num_read_kb":4528,"num_write":91,"num_write_kb":9656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"56'226","reported_seq":1476,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.188803+0000","last_change":"2024-09-18T00:57:39.056761+0000","last_active":"2024-09-18T00:58:20.188803+0000","last_peered":"2024-09-18T00:58:20.188803+0000","last_clean":"2024-09-18T00:58:20.188803+0000","last_became_active":"2024-09-18T00:57:36.910991+0000","last_became_peered":"2024-09-18T00:57:36.910991+0000","last_unstale":"2024-09-18T00:58:20.188803+0000","last_undegraded":"2024-09-18T00:58:20.188803+0000","last_fullsized":"2024-09-18T00:58:20.188803+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:14:49.504697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055104700000000004,"stat_sum":{"num_bytes":13501219,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":2230,"num_read_kb":10660,"num_write":267,"num_write_kb":8334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"56'312","reported_seq":978,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190752+0000","last_change":"2024-09-18T00:57:38.899783+0000","last_active":"2024-09-18T00:58:20.190752+0000","last_peered":"2024-09-18T00:58:20.190752+0000","last_clean":"2024-09-18T00:58:20.190752+0000","last_became_active":"2024-09-18T00:57:36.904999+0000","last_became_peered":"2024-09-18T00:57:36.904999+0000","last_unstale":"2024-09-18T00:58:20.190752+0000","last_undegraded":"2024-09-18T00:58:20.190752+0000","last_fullsized":"2024-09-18T00:58:20.190752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:11.047852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070399799999999995,"stat_sum":{"num_bytes":3670098,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":2124,"num_read_kb":5509,"num_write":405,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":523960440,"num_objects":1255,"num_object_clones":242,"num_object_copies":2510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1255,"num_whiteouts":620,"num_read":68061,"num_read_kb":332655,"num_write":7448,"num_write_kb":324766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":304402432,"data_stored":327295627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6482,"ondisk_log_size":6482,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67496,"kb_used_data":32152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94304344,"statfs":{"total":96636764160,"available":96567648256,"internally_reserved":0,"allocated":32923648,"data_stored":35278980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84540,"kb_used_data":45036,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94287300,"statfs":{"total":96636764160,"available":96550195200,"internally_reserved":0,"allocated":46116864,"data_stored":49793980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86276,"kb_used_data":46772,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94285564,"statfs":{"total":96636764160,"available":96548417536,"internally_reserved":0,"allocated":47894528,"data_stored":49580669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81944,"kb_used_data":35604,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94289896,"statfs":{"total":96636764160,"available":96552853504,"internally_reserved":0,"allocated":36458496,"data_stored":40803161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64388,"kb_used_data":28340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94307452,"statfs":{"total":96636764160,"available":96570830848,"internally_reserved":0,"allocated":29020160,"data_stored":31871717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89712,"kb_used_data":50188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94282128,"statfs":{"total":96636764160,"available":96544899072,"internally_reserved":0,"allocated":51392512,"data_stored":53392843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83656,"kb_used_data":44152,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94288184,"statfs":{"total":96636764160,"available":96551100416,"internally_reserved":0,"allocated":45211648,"data_stored":46441479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61068,"kb_used_data":21008,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94310772,"statfs":{"total":96636764160,"available":96574230528,"internally_reserved":0,"allocated":21512192,"data_stored":23294795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44703744,"data_stored":46285347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50159616,"data_stored":52536475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28438528,"data_stored":31656773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35852288,"data_stored":40561644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":47235072,"data_stored":49294521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":44896256,"data_stored":48947558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32276480,"data_stored":35003604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20840448,"data_stored":23009705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:58:25.647 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:58:25.661 INFO:teuthology.orchestra.run.smithi042.stdout:2223 punch from 0x7b47e9 to 0x7b7ea1, (0x36b8 bytes) 2024-09-18T00:58:25.661 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:25.661 INFO:teuthology.orchestra.run.smithi042.stdout:2226 trunc from 0xf9a316 to 0x1997fd4 2024-09-18T00:58:25.670 INFO:teuthology.orchestra.run.smithi042.stdout:2229 trunc from 0x1997fd4 to 0x33bd93 2024-09-18T00:58:25.682 INFO:teuthology.orchestra.run.smithi042.stdout:2230 read 0x26606d thru 0x26fd21 (0x9cb5 bytes) 2024-09-18T00:58:25.683 INFO:teuthology.orchestra.run.smithi042.stdout:2231 read 0xfcc8b thru 0x10b894 (0xec0a bytes) 2024-09-18T00:58:25.699 INFO:teuthology.orchestra.run.smithi042.stdout:2232 writesame 0x76b2fb2 thru 0x76b9f7e data_size 0x1(0x6fcd bytes) 2024-09-18T00:58:25.708 INFO:teuthology.orchestra.run.smithi042.stdout:2233 writesame 0x83b2931 thru 0x83b9851 data_size 0x57(0x6f21 bytes) 2024-09-18T00:58:25.708 INFO:teuthology.orchestra.run.smithi042.stdout:2234 read 0x7e9a53f thru 0x7ea75c9 (0xd08b bytes) 2024-09-18T00:58:25.709 INFO:teuthology.orchestra.run.smithi042.stdout:2236 read 0x6af3c61 thru 0x6af78a7 (0x3c47 bytes) 2024-09-18T00:58:25.709 INFO:teuthology.orchestra.run.smithi042.stdout:2237 write 0x73874ca thru 0x7388fec (0x1b23 bytes) 2024-09-18T00:58:25.725 INFO:teuthology.orchestra.run.smithi042.stdout:2238 writesame 0x95c5d0c thru 0x95cc285 data_size 0x1f(0x657a bytes) 2024-09-18T00:58:25.737 INFO:teuthology.orchestra.run.smithi042.stdout:2239 write 0xe390ef9 thru 0xe394718 (0x3820 bytes) 2024-09-18T00:58:25.744 INFO:teuthology.orchestra.run.smithi042.stdout:2242 read 0xdff3df4 thru 0xe003ad2 (0xfcdf bytes) 2024-09-18T00:58:25.744 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:25.737+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:25.745 INFO:teuthology.orchestra.run.smithi042.stdout:2243 writesame 0x8c309d6 thru 0x8c336db data_size 0x33(0x2d06 bytes) 2024-09-18T00:58:25.748 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:25.748 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:25.749 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:25.751 INFO:teuthology.orchestra.run.smithi042.stdout:2244 write 0x78ee2c7 thru 0x78efe8d (0x1bc7 bytes) 2024-09-18T00:58:25.757 INFO:teuthology.orchestra.run.smithi042.stdout:2245 read 0x8d92800 thru 0x8d99540 (0x6d41 bytes) 2024-09-18T00:58:25.759 INFO:teuthology.orchestra.run.smithi042.stdout:2246 read 0x4ca2fd3 thru 0x4ca3710 (0x73e bytes) 2024-09-18T00:58:25.759 INFO:teuthology.orchestra.run.smithi042.stdout:2247 read 0xa30a4e9 thru 0xa3157bf (0xb2d7 bytes) 2024-09-18T00:58:25.759 INFO:teuthology.orchestra.run.smithi042.stdout:2249 read 0x39fc9b5 thru 0x39fe7f1 (0x1e3d bytes) 2024-09-18T00:58:25.759 INFO:teuthology.orchestra.run.smithi042.stdout:2251 punch from 0x56e69ef to 0x56e8f86, (0x2597 bytes) 2024-09-18T00:58:25.759 INFO:teuthology.orchestra.run.smithi042.stdout:2252 read 0xb020677 thru 0xb02961c (0x8fa6 bytes) 2024-09-18T00:58:25.759 INFO:teuthology.orchestra.run.smithi042.stdout:2253 writesame 0xaa37cee thru 0xaa3c6df data_size 0xf(0x49f2 bytes) 2024-09-18T00:58:25.762 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:25.767 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:25.767 INFO:teuthology.orchestra.run.smithi042.stdout:2255 write 0xc481ba9 thru 0xc48b0a0 (0x94f8 bytes) 2024-09-18T00:58:25.776 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:25.776 INFO:teuthology.orchestra.run.smithi042.stdout:2257 punch from 0xd474e11 to 0xd47b983, (0x6b72 bytes) 2024-09-18T00:58:25.776 INFO:teuthology.orchestra.run.smithi042.stdout:2258 read 0x26a06e5 thru 0x26ab33a (0xac56 bytes) 2024-09-18T00:58:25.776 INFO:teuthology.orchestra.run.smithi042.stdout:2259 writesame 0xca5bf8a thru 0xca6bed9 data_size 0x5f(0xff50 bytes) 2024-09-18T00:58:25.790 INFO:teuthology.orchestra.run.smithi042.stdout:2261 writesame 0x6e42b43 thru 0x6e43292 data_size 0x6(0x750 bytes) 2024-09-18T00:58:25.809 INFO:teuthology.orchestra.run.smithi042.stdout:2263 trunc from 0xe394719 to 0x47ea608 2024-09-18T00:58:25.825 INFO:teuthology.orchestra.run.smithi042.stdout:2264 writesame 0x1113471 thru 0x111774a data_size 0x2(0x42da bytes) 2024-09-18T00:58:25.831 INFO:teuthology.orchestra.run.smithi042.stdout:2265 read 0x3a1beaf thru 0x3a2897e (0xcad0 bytes) 2024-09-18T00:58:25.831 INFO:teuthology.orchestra.run.smithi042.stdout:2267 write 0x9fc357 thru 0xa0967e (0xd328 bytes) 2024-09-18T00:58:25.836 INFO:teuthology.orchestra.run.smithi042.stdout:2268 trunc from 0x47ea608 to 0x5f57ff8 2024-09-18T00:58:25.845 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:25.842+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:25.856 INFO:teuthology.orchestra.run.smithi042.stdout:2269 write 0x9fa022c thru 0x9fad00a (0xcddf bytes) 2024-09-18T00:58:25.879 INFO:teuthology.orchestra.run.smithi042.stdout:2271 write 0xaae5e42 thru 0xaaf42f8 (0xe4b7 bytes) 2024-09-18T00:58:25.896 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.817939191000001, 2024-09-18T00:58:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:58:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:58:25.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.817009599999999, 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:58:25.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.816995540000001, 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:58:25.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:58:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:58:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:58:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:58:25.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.910 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.815704965000002, 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:58:25.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.815684492999999, 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:58:25.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.812890039999999, 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.920 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:25.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.812874856000001, 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:58:25.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:58:25.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.928 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.812851909999999, 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:58:25.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.808674271000001, 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:58:25.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:58:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:58:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:58:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.808653053999997, 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:58:25.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:58:25.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.808631671000001, 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.943 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.804731816999997, 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:58:25.947 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:58:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:58:25.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.804717615999998, 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:25.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.804699579000001, 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:58:25.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.804685513000003, 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:58:25.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:58:25.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.800660162, 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:58:25.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:25.969 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.800646149999999, 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:25.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:58:25.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.973 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.800632186000001, 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:58:25.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:58:25.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.038774232000002, 2024-09-18T00:58:25.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:58:25.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.913297861, 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:25.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:58:25.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:58:25.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:25.989 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:25.989 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:25.989 INFO:teuthology.orchestra.run.smithi042.stdout:2273 read 0x2458a22 thru 0x24627f9 (0x9dd8 bytes) 2024-09-18T00:58:25.990 INFO:teuthology.orchestra.run.smithi042.stdout:2274 writesame 0xe73f067 thru 0xe74c373 data_size 0x1(0xd30d bytes) 2024-09-18T00:58:25.990 INFO:teuthology.orchestra.run.smithi042.stdout:2275 writesame 0x4443770 thru 0x4450eb4 data_size 0x1(0xd745 bytes) 2024-09-18T00:58:25.991 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:25.991 INFO:teuthology.orchestra.run.smithi042.stdout:2276 writesame 0x3561249 thru 0x3568515 data_size 0x1(0x72cd bytes) 2024-09-18T00:58:25.992 INFO:teuthology.orchestra.run.smithi042.stdout:2277 read 0xb26258e thru 0xb264b89 (0x25fc bytes) 2024-09-18T00:58:25.992 INFO:teuthology.orchestra.run.smithi042.stdout:2278 writesame 0xe3a9eb3 thru 0xe3ae2cf data_size 0x7(0x441d bytes) 2024-09-18T00:58:25.992 INFO:teuthology.orchestra.run.smithi042.stdout:2280 trunc from 0xe74c374 to 0xa30b4fd 2024-09-18T00:58:25.993 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:58:25.993 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:58:26.001 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-09-18T00:58:25.897787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":524550827,"num_objects":1259,"num_object_clones":242,"num_object_copies":2518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1259,"num_whiteouts":620,"num_read":68177,"num_read_kb":332755,"num_write":7609,"num_write_kb":327184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6602,"ondisk_log_size":6602,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":619080,"kb_used_data":303252,"kb_used_omap":12,"kb_used_meta":307571,"kb_avail":754355640,"statfs":{"total":773094113280,"available":772460175360,"internally_reserved":0,"allocated":310530048,"data_stored":330457624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":314953264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":1472,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001080"},"pg_stats":[{"pgid":"3.1f","version":"56'174","reported_seq":1255,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.200336+0000","last_change":"2024-09-18T00:57:38.909849+0000","last_active":"2024-09-18T00:58:20.200336+0000","last_peered":"2024-09-18T00:58:20.200336+0000","last_clean":"2024-09-18T00:58:20.200336+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T00:58:20.200336+0000","last_undegraded":"2024-09-18T00:58:20.200336+0000","last_fullsized":"2024-09-18T00:58:20.200336+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:27:45.244510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001565786,"stat_sum":{"num_bytes":5807801,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":19,"num_read":2323,"num_read_kb":5553,"num_write":151,"num_write_kb":2637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"58'149","reported_seq":1227,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.659838+0000","last_change":"2024-09-18T00:57:38.908434+0000","last_active":"2024-09-18T00:58:20.659838+0000","last_peered":"2024-09-18T00:58:20.659838+0000","last_clean":"2024-09-18T00:58:20.659838+0000","last_became_active":"2024-09-18T00:57:36.897948+0000","last_became_peered":"2024-09-18T00:57:36.897948+0000","last_unstale":"2024-09-18T00:58:20.659838+0000","last_undegraded":"2024-09-18T00:58:20.659838+0000","last_fullsized":"2024-09-18T00:58:20.659838+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:44.221240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040574399999999998,"stat_sum":{"num_bytes":9427091,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":24,"num_read":1195,"num_read_kb":5732,"num_write":140,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"56'387","reported_seq":1238,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190770+0000","last_change":"2024-09-18T00:57:38.902294+0000","last_active":"2024-09-18T00:58:20.190770+0000","last_peered":"2024-09-18T00:58:20.190770+0000","last_clean":"2024-09-18T00:58:20.190770+0000","last_became_active":"2024-09-18T00:57:36.906790+0000","last_became_peered":"2024-09-18T00:57:36.906790+0000","last_unstale":"2024-09-18T00:58:20.190770+0000","last_undegraded":"2024-09-18T00:58:20.190770+0000","last_fullsized":"2024-09-18T00:58:20.190770+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:58:37.920323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039089,"stat_sum":{"num_bytes":10473306,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":24,"num_read":2271,"num_read_kb":12736,"num_write":591,"num_write_kb":2130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"56'314","reported_seq":1567,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.186931+0000","last_change":"2024-09-18T00:57:38.901600+0000","last_active":"2024-09-18T00:58:20.186931+0000","last_peered":"2024-09-18T00:58:20.186931+0000","last_clean":"2024-09-18T00:58:20.186931+0000","last_became_active":"2024-09-18T00:57:36.906003+0000","last_became_peered":"2024-09-18T00:57:36.906003+0000","last_unstale":"2024-09-18T00:58:20.186931+0000","last_undegraded":"2024-09-18T00:58:20.186931+0000","last_fullsized":"2024-09-18T00:58:20.186931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:39:14.537556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000711266,"stat_sum":{"num_bytes":4672646,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":2555,"num_read_kb":8820,"num_write":370,"num_write_kb":1830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"58'342","reported_seq":5404,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.661429+0000","last_change":"2024-09-18T00:57:38.921854+0000","last_active":"2024-09-18T00:58:20.661429+0000","last_peered":"2024-09-18T00:58:20.661429+0000","last_clean":"2024-09-18T00:58:20.661429+0000","last_became_active":"2024-09-18T00:57:36.913393+0000","last_became_peered":"2024-09-18T00:57:36.913393+0000","last_unstale":"2024-09-18T00:58:20.661429+0000","last_undegraded":"2024-09-18T00:58:20.661429+0000","last_fullsized":"2024-09-18T00:58:20.661429+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:35:13.924235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002554022,"stat_sum":{"num_bytes":28064553,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":6009,"num_read_kb":16867,"num_write":481,"num_write_kb":11875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"58'168","reported_seq":1695,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.822413+0000","last_change":"2024-09-18T00:57:38.901805+0000","last_active":"2024-09-18T00:58:20.822413+0000","last_peered":"2024-09-18T00:58:20.822413+0000","last_clean":"2024-09-18T00:58:20.822413+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T00:58:20.822413+0000","last_undegraded":"2024-09-18T00:58:20.822413+0000","last_fullsized":"2024-09-18T00:58:20.822413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:13.529614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098070899999999996,"stat_sum":{"num_bytes":26466557,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":17,"num_read":2384,"num_read_kb":8111,"num_write":136,"num_write_kb":764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"58'160","reported_seq":8632,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.879595+0000","last_change":"2024-09-18T00:57:38.922010+0000","last_active":"2024-09-18T00:58:20.879595+0000","last_peered":"2024-09-18T00:58:20.879595+0000","last_clean":"2024-09-18T00:58:20.879595+0000","last_became_active":"2024-09-18T00:57:36.910457+0000","last_became_peered":"2024-09-18T00:57:36.910457+0000","last_unstale":"2024-09-18T00:58:20.879595+0000","last_undegraded":"2024-09-18T00:58:20.879595+0000","last_fullsized":"2024-09-18T00:58:20.879595+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:46:12.174403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026842849999999998,"stat_sum":{"num_bytes":34158112,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":8465,"num_read_kb":26532,"num_write":154,"num_write_kb":13870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"58'370","reported_seq":1321,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.907359+0000","last_change":"2024-09-18T00:57:38.908144+0000","last_active":"2024-09-18T00:58:20.907359+0000","last_peered":"2024-09-18T00:58:20.907359+0000","last_clean":"2024-09-18T00:58:20.907359+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T00:58:20.907359+0000","last_undegraded":"2024-09-18T00:58:20.907359+0000","last_fullsized":"2024-09-18T00:58:20.907359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:04:12.939208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000871597,"stat_sum":{"num_bytes":29218606,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":2455,"num_read_kb":3397,"num_write":504,"num_write_kb":34655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"58'159","reported_seq":841,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.833465+0000","last_change":"2024-09-18T00:57:38.910038+0000","last_active":"2024-09-18T00:58:20.833465+0000","last_peered":"2024-09-18T00:58:20.833465+0000","last_clean":"2024-09-18T00:58:20.833465+0000","last_became_active":"2024-09-18T00:57:36.895915+0000","last_became_peered":"2024-09-18T00:57:36.895915+0000","last_unstale":"2024-09-18T00:58:20.833465+0000","last_undegraded":"2024-09-18T00:58:20.833465+0000","last_fullsized":"2024-09-18T00:58:20.833465+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:25.225803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001269769,"stat_sum":{"num_bytes":13162855,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":999,"num_read_kb":12929,"num_write":150,"num_write_kb":12595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"58'212","reported_seq":2845,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.649324+0000","last_change":"2024-09-18T00:57:39.381845+0000","last_active":"2024-09-18T00:58:20.649324+0000","last_peered":"2024-09-18T00:58:20.649324+0000","last_clean":"2024-09-18T00:58:20.649324+0000","last_became_active":"2024-09-18T00:57:36.894538+0000","last_became_peered":"2024-09-18T00:57:36.894538+0000","last_unstale":"2024-09-18T00:58:20.649324+0000","last_undegraded":"2024-09-18T00:58:20.649324+0000","last_fullsized":"2024-09-18T00:58:20.649324+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:58.064925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076552,"stat_sum":{"num_bytes":12181857,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":3140,"num_read_kb":14578,"num_write":335,"num_write_kb":16391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"56'123","reported_seq":754,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.216036+0000","last_change":"2024-09-18T00:57:38.909647+0000","last_active":"2024-09-18T00:58:20.216036+0000","last_peered":"2024-09-18T00:58:20.216036+0000","last_clean":"2024-09-18T00:58:20.216036+0000","last_became_active":"2024-09-18T00:57:36.902977+0000","last_became_peered":"2024-09-18T00:57:36.902977+0000","last_unstale":"2024-09-18T00:58:20.216036+0000","last_undegraded":"2024-09-18T00:58:20.216036+0000","last_fullsized":"2024-09-18T00:58:20.216036+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:56:23.484909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051351900000000004,"stat_sum":{"num_bytes":23986238,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":571,"num_read_kb":7729,"num_write":146,"num_write_kb":38402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"58'88","reported_seq":507,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.649599+0000","last_change":"2024-09-18T00:57:38.902578+0000","last_active":"2024-09-18T00:58:20.649599+0000","last_peered":"2024-09-18T00:58:20.649599+0000","last_clean":"2024-09-18T00:58:20.649599+0000","last_became_active":"2024-09-18T00:57:36.894977+0000","last_became_peered":"2024-09-18T00:57:36.894977+0000","last_unstale":"2024-09-18T00:58:20.649599+0000","last_undegraded":"2024-09-18T00:58:20.649599+0000","last_fullsized":"2024-09-18T00:58:20.649599+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:35:55.273404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028223319999999999,"stat_sum":{"num_bytes":5890208,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":357,"num_read_kb":762,"num_write":69,"num_write_kb":846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.191066+0000","last_change":"2024-09-18T00:57:29.328163+0000","last_active":"2024-09-18T00:58:20.191066+0000","last_peered":"2024-09-18T00:58:20.191066+0000","last_clean":"2024-09-18T00:58:20.191066+0000","last_became_active":"2024-09-18T00:57:27.311231+0000","last_became_peered":"2024-09-18T00:57:27.311231+0000","last_unstale":"2024-09-18T00:58:20.191066+0000","last_undegraded":"2024-09-18T00:58:20.191066+0000","last_fullsized":"2024-09-18T00:58:20.191066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:57.340012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020990940000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"56'195","reported_seq":1350,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.209670+0000","last_change":"2024-09-18T00:57:38.920098+0000","last_active":"2024-09-18T00:58:20.209670+0000","last_peered":"2024-09-18T00:58:20.209670+0000","last_clean":"2024-09-18T00:58:20.209670+0000","last_became_active":"2024-09-18T00:57:36.920599+0000","last_became_peered":"2024-09-18T00:57:36.920599+0000","last_unstale":"2024-09-18T00:58:20.209670+0000","last_undegraded":"2024-09-18T00:58:20.209670+0000","last_fullsized":"2024-09-18T00:58:20.209670+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:25:08.740023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276634,"stat_sum":{"num_bytes":10760753,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":21,"num_read":1685,"num_read_kb":14337,"num_write":262,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.209497+0000","last_change":"2024-09-18T00:57:29.327841+0000","last_active":"2024-09-18T00:58:20.209497+0000","last_peered":"2024-09-18T00:58:20.209497+0000","last_clean":"2024-09-18T00:58:20.209497+0000","last_became_active":"2024-09-18T00:57:27.315417+0000","last_became_peered":"2024-09-18T00:57:27.315417+0000","last_unstale":"2024-09-18T00:58:20.209497+0000","last_undegraded":"2024-09-18T00:58:20.209497+0000","last_fullsized":"2024-09-18T00:58:20.209497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:54.335334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"58'177","reported_seq":3690,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.652702+0000","last_change":"2024-09-18T00:57:38.920634+0000","last_active":"2024-09-18T00:58:20.652702+0000","last_peered":"2024-09-18T00:58:20.652702+0000","last_clean":"2024-09-18T00:58:20.652702+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T00:58:20.652702+0000","last_undegraded":"2024-09-18T00:58:20.652702+0000","last_fullsized":"2024-09-18T00:58:20.652702+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:20.716204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728301,"stat_sum":{"num_bytes":15265512,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":4180,"num_read_kb":9337,"num_write":144,"num_write_kb":6194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190971+0000","last_change":"2024-09-18T00:57:29.327995+0000","last_active":"2024-09-18T00:58:20.190971+0000","last_peered":"2024-09-18T00:58:20.190971+0000","last_clean":"2024-09-18T00:58:20.190971+0000","last_became_active":"2024-09-18T00:57:27.316238+0000","last_became_peered":"2024-09-18T00:57:27.316238+0000","last_unstale":"2024-09-18T00:58:20.190971+0000","last_undegraded":"2024-09-18T00:58:20.190971+0000","last_fullsized":"2024-09-18T00:58:20.190971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019441149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"58'123","reported_seq":711,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.875857+0000","last_change":"2024-09-18T00:57:39.056519+0000","last_active":"2024-09-18T00:58:20.875857+0000","last_peered":"2024-09-18T00:58:20.875857+0000","last_clean":"2024-09-18T00:58:20.875857+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T00:58:20.875857+0000","last_undegraded":"2024-09-18T00:58:20.875857+0000","last_fullsized":"2024-09-18T00:58:20.875857+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:01.547712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043106399999999999,"stat_sum":{"num_bytes":18056261,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":544,"num_read_kb":5637,"num_write":103,"num_write_kb":9328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.200354+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:58:20.200354+0000","last_peered":"2024-09-18T00:58:20.200354+0000","last_clean":"2024-09-18T00:58:20.200354+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:58:20.200354+0000","last_undegraded":"2024-09-18T00:58:20.200354+0000","last_fullsized":"2024-09-18T00:58:20.200354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"58'110","reported_seq":715,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.650546+0000","last_change":"2024-09-18T00:57:39.381374+0000","last_active":"2024-09-18T00:58:20.650546+0000","last_peered":"2024-09-18T00:58:20.650546+0000","last_clean":"2024-09-18T00:58:20.650546+0000","last_became_active":"2024-09-18T00:57:36.902781+0000","last_became_peered":"2024-09-18T00:57:36.902781+0000","last_unstale":"2024-09-18T00:58:20.650546+0000","last_undegraded":"2024-09-18T00:58:20.650546+0000","last_fullsized":"2024-09-18T00:58:20.650546+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:10:22.436994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000584944,"stat_sum":{"num_bytes":8305362,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":826,"num_read_kb":10701,"num_write":95,"num_write_kb":1315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"58'235","reported_seq":1205,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.334598+0000","last_change":"2024-09-18T00:57:38.909273+0000","last_active":"2024-09-18T00:58:20.334598+0000","last_peered":"2024-09-18T00:58:20.334598+0000","last_clean":"2024-09-18T00:58:20.334598+0000","last_became_active":"2024-09-18T00:57:36.906264+0000","last_became_peered":"2024-09-18T00:57:36.906264+0000","last_unstale":"2024-09-18T00:58:20.334598+0000","last_undegraded":"2024-09-18T00:58:20.334598+0000","last_fullsized":"2024-09-18T00:58:20.334598+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:29.318187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056105599999999999,"stat_sum":{"num_bytes":25332204,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":1676,"num_read_kb":2036,"num_write":336,"num_write_kb":15885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190674+0000","last_change":"2024-09-18T00:57:29.327436+0000","last_active":"2024-09-18T00:58:20.190674+0000","last_peered":"2024-09-18T00:58:20.190674+0000","last_clean":"2024-09-18T00:58:20.190674+0000","last_became_active":"2024-09-18T00:57:27.410600+0000","last_became_peered":"2024-09-18T00:57:27.410600+0000","last_unstale":"2024-09-18T00:58:20.190674+0000","last_undegraded":"2024-09-18T00:58:20.190674+0000","last_fullsized":"2024-09-18T00:58:20.190674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:59.380173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"58'224","reported_seq":4789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.285564+0000","last_change":"2024-09-18T00:57:38.907753+0000","last_active":"2024-09-18T00:58:20.285564+0000","last_peered":"2024-09-18T00:58:20.285564+0000","last_clean":"2024-09-18T00:58:20.285564+0000","last_became_active":"2024-09-18T00:57:36.907660+0000","last_became_peered":"2024-09-18T00:57:36.907660+0000","last_unstale":"2024-09-18T00:58:20.285564+0000","last_undegraded":"2024-09-18T00:58:20.285564+0000","last_fullsized":"2024-09-18T00:58:20.285564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:20:33.569644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090282999999999997,"stat_sum":{"num_bytes":14224233,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":25,"num_read":5032,"num_read_kb":13060,"num_write":275,"num_write_kb":3505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":93,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T00:58:22.286852+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:58:22.286852+0000","last_peered":"2024-09-18T00:58:22.286852+0000","last_clean":"2024-09-18T00:58:22.286852+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:58:22.286852+0000","last_undegraded":"2024-09-18T00:58:22.286852+0000","last_fullsized":"2024-09-18T00:58:22.286852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"58'173","reported_seq":1633,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.849849+0000","last_change":"2024-09-18T00:57:39.056604+0000","last_active":"2024-09-18T00:58:20.849849+0000","last_peered":"2024-09-18T00:58:20.849849+0000","last_clean":"2024-09-18T00:58:20.849849+0000","last_became_active":"2024-09-18T00:57:36.903081+0000","last_became_peered":"2024-09-18T00:57:36.903081+0000","last_unstale":"2024-09-18T00:58:20.849849+0000","last_undegraded":"2024-09-18T00:58:20.849849+0000","last_fullsized":"2024-09-18T00:58:20.849849+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:20.963144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045206800000000002,"stat_sum":{"num_bytes":10082907,"num_objects":45,"num_object_clones":9,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":2074,"num_read_kb":13164,"num_write":146,"num_write_kb":1232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190411+0000","last_change":"2024-09-18T00:57:29.327276+0000","last_active":"2024-09-18T00:58:20.190411+0000","last_peered":"2024-09-18T00:58:20.190411+0000","last_clean":"2024-09-18T00:58:20.190411+0000","last_became_active":"2024-09-18T00:57:27.316217+0000","last_became_peered":"2024-09-18T00:57:27.316217+0000","last_unstale":"2024-09-18T00:58:20.190411+0000","last_undegraded":"2024-09-18T00:58:20.190411+0000","last_fullsized":"2024-09-18T00:58:20.190411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:38.026720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012774469999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"56'125","reported_seq":521,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.209289+0000","last_change":"2024-09-18T00:57:38.919998+0000","last_active":"2024-09-18T00:58:20.209289+0000","last_peered":"2024-09-18T00:58:20.209289+0000","last_clean":"2024-09-18T00:58:20.209289+0000","last_became_active":"2024-09-18T00:57:36.906680+0000","last_became_peered":"2024-09-18T00:57:36.906680+0000","last_unstale":"2024-09-18T00:58:20.209289+0000","last_undegraded":"2024-09-18T00:58:20.209289+0000","last_fullsized":"2024-09-18T00:58:20.209289+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:16.072424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064609,"stat_sum":{"num_bytes":14095436,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":24,"num_read":360,"num_read_kb":2311,"num_write":103,"num_write_kb":33816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190856+0000","last_change":"2024-09-18T00:57:29.328733+0000","last_active":"2024-09-18T00:58:20.190856+0000","last_peered":"2024-09-18T00:58:20.190856+0000","last_clean":"2024-09-18T00:58:20.190856+0000","last_became_active":"2024-09-18T00:57:27.410396+0000","last_became_peered":"2024-09-18T00:57:27.410396+0000","last_unstale":"2024-09-18T00:58:20.190856+0000","last_undegraded":"2024-09-18T00:58:20.190856+0000","last_fullsized":"2024-09-18T00:58:20.190856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:27.438117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":240,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.196852+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:58:20.196852+0000","last_peered":"2024-09-18T00:58:20.196852+0000","last_clean":"2024-09-18T00:58:20.196852+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:58:20.196852+0000","last_undegraded":"2024-09-18T00:58:20.196852+0000","last_fullsized":"2024-09-18T00:58:20.196852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"58'149","reported_seq":1084,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.913643+0000","last_change":"2024-09-18T00:57:38.919534+0000","last_active":"2024-09-18T00:58:20.913643+0000","last_peered":"2024-09-18T00:58:20.913643+0000","last_clean":"2024-09-18T00:58:20.913643+0000","last_became_active":"2024-09-18T00:57:36.908955+0000","last_became_peered":"2024-09-18T00:57:36.908955+0000","last_unstale":"2024-09-18T00:58:20.913643+0000","last_undegraded":"2024-09-18T00:58:20.913643+0000","last_fullsized":"2024-09-18T00:58:20.913643+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:40.118305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077760500000000001,"stat_sum":{"num_bytes":31528837,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":21,"num_read":972,"num_read_kb":14955,"num_write":155,"num_write_kb":26152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"58'121","reported_seq":780,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.902028+0000","last_change":"2024-09-18T00:57:38.899513+0000","last_active":"2024-09-18T00:58:20.902028+0000","last_peered":"2024-09-18T00:58:20.902028+0000","last_clean":"2024-09-18T00:58:20.902028+0000","last_became_active":"2024-09-18T00:57:36.911337+0000","last_became_peered":"2024-09-18T00:57:36.911337+0000","last_unstale":"2024-09-18T00:58:20.902028+0000","last_undegraded":"2024-09-18T00:58:20.902028+0000","last_fullsized":"2024-09-18T00:58:20.902028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:22:40.060688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061570699999999995,"stat_sum":{"num_bytes":15290136,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":18,"num_read":600,"num_read_kb":3357,"num_write":105,"num_write_kb":16064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"58'138","reported_seq":879,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.659321+0000","last_change":"2024-09-18T00:57:38.899904+0000","last_active":"2024-09-18T00:58:20.659321+0000","last_peered":"2024-09-18T00:58:20.659321+0000","last_clean":"2024-09-18T00:58:20.659321+0000","last_became_active":"2024-09-18T00:57:36.901837+0000","last_became_peered":"2024-09-18T00:57:36.901837+0000","last_unstale":"2024-09-18T00:58:20.659321+0000","last_undegraded":"2024-09-18T00:58:20.659321+0000","last_fullsized":"2024-09-18T00:58:20.659321+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:10.662378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097047300000000002,"stat_sum":{"num_bytes":11558357,"num_objects":43,"num_object_clones":5,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":718,"num_read_kb":12136,"num_write":113,"num_write_kb":8906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"60'122","reported_seq":1869,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T00:58:22.568467+0000","last_change":"2024-09-18T00:57:38.899763+0000","last_active":"2024-09-18T00:58:22.568467+0000","last_peered":"2024-09-18T00:58:22.568467+0000","last_clean":"2024-09-18T00:58:22.568467+0000","last_became_active":"2024-09-18T00:57:36.905869+0000","last_became_peered":"2024-09-18T00:57:36.905869+0000","last_unstale":"2024-09-18T00:58:22.568467+0000","last_undegraded":"2024-09-18T00:58:22.568467+0000","last_fullsized":"2024-09-18T00:58:22.568467+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:49.461946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097615600000000005,"stat_sum":{"num_bytes":12997052,"num_objects":51,"num_object_clones":15,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":19,"num_read":1888,"num_read_kb":24202,"num_write":84,"num_write_kb":2441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"58'266","reported_seq":2090,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.871483+0000","last_change":"2024-09-18T00:57:38.909165+0000","last_active":"2024-09-18T00:58:20.871483+0000","last_peered":"2024-09-18T00:58:20.871483+0000","last_clean":"2024-09-18T00:58:20.871483+0000","last_became_active":"2024-09-18T00:57:36.908288+0000","last_became_peered":"2024-09-18T00:57:36.908288+0000","last_unstale":"2024-09-18T00:58:20.871483+0000","last_undegraded":"2024-09-18T00:58:20.871483+0000","last_fullsized":"2024-09-18T00:58:20.871483+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:19:16.196155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044375399999999998,"stat_sum":{"num_bytes":16662412,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":2609,"num_read_kb":9294,"num_write":335,"num_write_kb":4553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"58'490","reported_seq":1492,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.883177+0000","last_change":"2024-09-18T00:57:38.903046+0000","last_active":"2024-09-18T00:58:20.883177+0000","last_peered":"2024-09-18T00:58:20.883177+0000","last_clean":"2024-09-18T00:58:20.883177+0000","last_became_active":"2024-09-18T00:57:36.900286+0000","last_became_peered":"2024-09-18T00:57:36.900286+0000","last_unstale":"2024-09-18T00:58:20.883177+0000","last_undegraded":"2024-09-18T00:58:20.883177+0000","last_fullsized":"2024-09-18T00:58:20.883177+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:33.280360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023201229999999999,"stat_sum":{"num_bytes":14935651,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":23,"num_read":3439,"num_read_kb":10352,"num_write":669,"num_write_kb":3385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"58'148","reported_seq":540,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.771776+0000","last_change":"2024-09-18T00:57:39.382152+0000","last_active":"2024-09-18T00:58:20.771776+0000","last_peered":"2024-09-18T00:58:20.771776+0000","last_clean":"2024-09-18T00:58:20.771776+0000","last_became_active":"2024-09-18T00:57:36.904791+0000","last_became_peered":"2024-09-18T00:57:36.904791+0000","last_unstale":"2024-09-18T00:58:20.771776+0000","last_undegraded":"2024-09-18T00:58:20.771776+0000","last_fullsized":"2024-09-18T00:58:20.771776+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:53:16.259577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090629000000000003,"stat_sum":{"num_bytes":8037756,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":700,"num_read_kb":2062,"num_write":124,"num_write_kb":762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"58'199","reported_seq":568,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.885793+0000","last_change":"2024-09-18T00:57:38.909597+0000","last_active":"2024-09-18T00:58:20.885793+0000","last_peered":"2024-09-18T00:58:20.885793+0000","last_clean":"2024-09-18T00:58:20.885793+0000","last_became_active":"2024-09-18T00:57:36.905462+0000","last_became_peered":"2024-09-18T00:57:36.905462+0000","last_unstale":"2024-09-18T00:58:20.885793+0000","last_undegraded":"2024-09-18T00:58:20.885793+0000","last_fullsized":"2024-09-18T00:58:20.885793+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:59:17.641292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081831799999999995,"stat_sum":{"num_bytes":23466562,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":820,"num_read_kb":6727,"num_write":248,"num_write_kb":6274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"58'191","reported_seq":2019,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.489364+0000","last_change":"2024-09-18T00:57:38.900906+0000","last_active":"2024-09-18T00:58:20.489364+0000","last_peered":"2024-09-18T00:58:20.489364+0000","last_clean":"2024-09-18T00:58:20.489364+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T00:58:20.489364+0000","last_undegraded":"2024-09-18T00:58:20.489364+0000","last_fullsized":"2024-09-18T00:58:20.489364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:14:20.481320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069386899999999995,"stat_sum":{"num_bytes":32329174,"num_objects":58,"num_object_clones":13,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":2404,"num_read_kb":34544,"num_write":161,"num_write_kb":24139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"58'112","reported_seq":628,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.832827+0000","last_change":"2024-09-18T00:57:38.899297+0000","last_active":"2024-09-18T00:58:20.832827+0000","last_peered":"2024-09-18T00:58:20.832827+0000","last_clean":"2024-09-18T00:58:20.832827+0000","last_became_active":"2024-09-18T00:57:36.897289+0000","last_became_peered":"2024-09-18T00:57:36.897289+0000","last_unstale":"2024-09-18T00:58:20.832827+0000","last_undegraded":"2024-09-18T00:58:20.832827+0000","last_fullsized":"2024-09-18T00:58:20.832827+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:24.696611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071487799999999995,"stat_sum":{"num_bytes":20350688,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":456,"num_read_kb":4528,"num_write":91,"num_write_kb":9656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"56'226","reported_seq":1476,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.188803+0000","last_change":"2024-09-18T00:57:39.056761+0000","last_active":"2024-09-18T00:58:20.188803+0000","last_peered":"2024-09-18T00:58:20.188803+0000","last_clean":"2024-09-18T00:58:20.188803+0000","last_became_active":"2024-09-18T00:57:36.910991+0000","last_became_peered":"2024-09-18T00:57:36.910991+0000","last_unstale":"2024-09-18T00:58:20.188803+0000","last_undegraded":"2024-09-18T00:58:20.188803+0000","last_fullsized":"2024-09-18T00:58:20.188803+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:14:49.504697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055104700000000004,"stat_sum":{"num_bytes":13501219,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":2230,"num_read_kb":10660,"num_write":267,"num_write_kb":8334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"56'312","reported_seq":978,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-18T00:58:20.190752+0000","last_change":"2024-09-18T00:57:38.899783+0000","last_active":"2024-09-18T00:58:20.190752+0000","last_peered":"2024-09-18T00:58:20.190752+0000","last_clean":"2024-09-18T00:58:20.190752+0000","last_became_active":"2024-09-18T00:57:36.904999+0000","last_became_peered":"2024-09-18T00:57:36.904999+0000","last_unstale":"2024-09-18T00:58:20.190752+0000","last_undegraded":"2024-09-18T00:58:20.190752+0000","last_fullsized":"2024-09-18T00:58:20.190752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:11.047852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070399799999999995,"stat_sum":{"num_bytes":3670098,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":2124,"num_read_kb":5509,"num_write":405,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":523960440,"num_objects":1255,"num_object_clones":242,"num_object_copies":2510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1255,"num_whiteouts":620,"num_read":68061,"num_read_kb":332655,"num_write":7448,"num_write_kb":324766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":304402432,"data_stored":327295627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6482,"ondisk_log_size":6482,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67496,"kb_used_data":32152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94304344,"statfs":{"total":96636764160,"available":96567648256,"internally_reserved":0,"allocated":32923648,"data_stored":35278980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84540,"kb_used_data":45036,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94287300,"statfs":{"total":96636764160,"available":96550195200,"internally_reserved":0,"allocated":46116864,"data_stored":49793980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86276,"kb_used_data":46772,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94285564,"statfs":{"total":96636764160,"available":96548417536,"internally_reserved":0,"allocated":47894528,"data_stored":49580669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81944,"kb_used_data":35604,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94289896,"statfs":{"total":96636764160,"available":96552853504,"internally_reserved":0,"allocated":36458496,"data_stored":40803161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64388,"kb_used_data":28340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94307452,"statfs":{"total":96636764160,"available":96570830848,"internally_reserved":0,"allocated":29020160,"data_stored":31871717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89712,"kb_used_data":50188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94282128,"statfs":{"total":96636764160,"available":96544899072,"internally_reserved":0,"allocated":51392512,"data_stored":53392843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83656,"kb_used_data":44152,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94288184,"statfs":{"total":96636764160,"available":96551100416,"internally_reserved":0,"allocated":45211648,"data_stored":46441479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61068,"kb_used_data":21008,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94310772,"statfs":{"total":96636764160,"available":96574230528,"internally_reserved":0,"allocated":21512192,"data_stored":23294795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44703744,"data_stored":46285347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50159616,"data_stored":52536475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28438528,"data_stored":31656773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35852288,"data_stored":40561644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":47235072,"data_stored":49294521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":44896256,"data_stored":48947558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32276480,"data_stored":35003604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20840448,"data_stored":23009705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:58:26.003 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T00:58:26.003 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:25.994+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:26.004 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:58:26.004 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-18T00:58:26.004 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T00:58:26.004 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T00:58:26.004 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-18T00:58:26.006 INFO:teuthology.orchestra.run.smithi042.stdout:2281 write 0xa67050d thru 0xa67e0e7 (0xdbdb bytes) 2024-09-18T00:58:26.018 INFO:teuthology.orchestra.run.smithi042.stdout:2282 write 0xa8214c4 thru 0xa82ca1b (0xb558 bytes) 2024-09-18T00:58:26.023 INFO:teuthology.orchestra.run.smithi042.stdout:2283 read 0x63e35c9 thru 0x63e6c6f (0x36a7 bytes) 2024-09-18T00:58:26.023 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.023 INFO:teuthology.orchestra.run.smithi042.stdout:2287 read 0x9a93562 thru 0x9a9dfe6 (0xaa85 bytes) 2024-09-18T00:58:26.095 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:26.089+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:26.116 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:26.116 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:26.117 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "" 2024-09-18T00:58:26.117 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:26.132 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:26.197 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:26.189+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:26.223 INFO:teuthology.orchestra.run.smithi042.stdout:2288 write 0x32e6c11 thru 0x32f3e68 (0xd258 bytes) 2024-09-18T00:58:26.239 INFO:teuthology.orchestra.run.smithi042.stdout:2290 read 0x975e4ef thru 0x975e93e (0x450 bytes) 2024-09-18T00:58:26.239 INFO:teuthology.orchestra.run.smithi042.stdout:2292 read 0xa1b4aa6 thru 0xa1ba028 (0x5583 bytes) 2024-09-18T00:58:26.240 INFO:teuthology.orchestra.run.smithi042.stdout:2293 punch from 0x3572a8c to 0x3581e02, (0xf376 bytes) 2024-09-18T00:58:26.240 INFO:teuthology.orchestra.run.smithi042.stdout:2295 write 0x960dd8 thru 0x9618d5 (0xafe bytes) 2024-09-18T00:58:26.244 INFO:teuthology.orchestra.run.smithi042.stdout:2296 read 0x232651b thru 0x2330a50 (0xa536 bytes) 2024-09-18T00:58:26.244 INFO:teuthology.orchestra.run.smithi042.stdout:2301 read 0x364ab34 thru 0x364dba3 (0x3070 bytes) 2024-09-18T00:58:26.252 INFO:teuthology.orchestra.run.smithi042.stdout:2303 write 0xab52d2a thru 0xab613b0 (0xe687 bytes) 2024-09-18T00:58:26.258 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:58:26.270 INFO:teuthology.orchestra.run.smithi042.stdout:2304 write 0xbb2acd1 thru 0xbb2ccb5 (0x1fe5 bytes) 2024-09-18T00:58:26.277 INFO:teuthology.orchestra.run.smithi042.stdout:2305 write 0x6bcb575 thru 0x6bd19ad (0x6439 bytes) 2024-09-18T00:58:26.297 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:26.297 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:26.297 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:26.297 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:26.297 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:26.298 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:26.294+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:26.299 INFO:teuthology.orchestra.run.smithi042.stdout:2307 trunc from 0xbb2ccb6 to 0x30a83a2 2024-09-18T00:58:26.315 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:26.325 INFO:teuthology.orchestra.run.smithi042.stdout:2308 punch from 0xfa9471 to 0xfb16a2, (0x8231 bytes) 2024-09-18T00:58:26.341 INFO:teuthology.orchestra.run.smithi042.stdout:2310 writesame 0x9b2fabe thru 0x9b3e339 data_size 0x2(0xe87c bytes) 2024-09-18T00:58:26.341 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.352 INFO:teuthology.orchestra.run.smithi042.stdout:2312 write 0xd053351 thru 0xd05edb6 (0xba66 bytes) 2024-09-18T00:58:26.366 INFO:teuthology.orchestra.run.smithi042.stdout:2313 write 0x5676030 thru 0x5682d31 (0xcd02 bytes) 2024-09-18T00:58:26.379 INFO:teuthology.orchestra.run.smithi042.stdout:2314 punch from 0x10638ed to 0x1066e9e, (0x35b1 bytes) 2024-09-18T00:58:26.379 INFO:teuthology.orchestra.run.smithi042.stdout:2315 read 0xb9c2597 thru 0xb9ca004 (0x7a6e bytes) 2024-09-18T00:58:26.379 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.380 INFO:teuthology.orchestra.run.smithi042.stdout:2318 write 0x5721768 thru 0x572e32f (0xcbc8 bytes) 2024-09-18T00:58:26.385 INFO:teuthology.orchestra.run.smithi042.stdout:2319 write 0x45d8462 thru 0x45d8b9e (0x73d bytes) 2024-09-18T00:58:26.390 INFO:teuthology.orchestra.run.smithi042.stdout:2320 writesame 0xb52972 thru 0xb6062d data_size 0x66(0xdcbc bytes) 2024-09-18T00:58:26.394 INFO:teuthology.orchestra.run.smithi042.stdout:2321 writesame 0x9d9904 thru 0x9dd478 data_size 0x1(0x3b75 bytes) 2024-09-18T00:58:26.399 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:26.393+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:26.402 INFO:teuthology.orchestra.run.smithi042.stdout:2323 writesame 0x4d35160 thru 0x4d37c7f data_size 0x3c(0x2b20 bytes) 2024-09-18T00:58:26.404 INFO:teuthology.orchestra.run.smithi042.stdout:2324 read 0x5b14ab0 thru 0x5b18742 (0x3c93 bytes) 2024-09-18T00:58:26.405 INFO:teuthology.orchestra.run.smithi042.stdout:2326 read 0x571f045 thru 0x572d63d (0xe5f9 bytes) 2024-09-18T00:58:26.406 INFO:teuthology.orchestra.run.smithi042.stdout:2328 writesame 0x7decc54 thru 0x7deed1e data_size 0x17(0x20cb bytes) 2024-09-18T00:58:26.412 INFO:teuthology.orchestra.run.smithi042.stdout:2329 writesame 0x2846651 thru 0x2849637 data_size 0x1(0x2fe7 bytes) 2024-09-18T00:58:26.435 INFO:teuthology.orchestra.run.smithi042.stdout:2330 trunc from 0xd05edb7 to 0x7a2fe4e 2024-09-18T00:58:26.452 INFO:teuthology.orchestra.run.smithi042.stdout:2331 read 0x136fcbd thru 0x1370d89 (0x10cd bytes) 2024-09-18T00:58:26.453 INFO:teuthology.orchestra.run.smithi042.stdout:2332 read 0x80e4c4 thru 0x8107e5 (0x2322 bytes) 2024-09-18T00:58:26.455 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:26.455 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:26.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:26.455 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:26.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.455 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:58:26.455 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:26.455 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.139351161999997, 2024-09-18T00:58:26.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101264911, 2024-09-18T00:58:26.456 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.456 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:26.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4226", 2024-09-18T00:58:26.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2155402750", 2024-09-18T00:58:26.456 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:26.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.456 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313760+0000", 2024-09-18T00:58:26.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.612e-06 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313768+0000", 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9249999999999995e-06 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313786+0000", 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8253999999999998e-05 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313803+0000", 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313935+0000", 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013180399999999999 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313954+0000", 2024-09-18T00:58:26.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8768999999999999e-05 2024-09-18T00:58:26.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.411874+0000", 2024-09-18T00:58:26.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.097920646 2024-09-18T00:58:26.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412292+0000", 2024-09-18T00:58:26.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041777699999999998 2024-09-18T00:58:26.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:26.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412675+0000", 2024-09-18T00:58:26.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038262300000000001 2024-09-18T00:58:26.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:26.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414019+0000", 2024-09-18T00:58:26.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013436699999999999 2024-09-18T00:58:26.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:26.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414872+0000", 2024-09-18T00:58:26.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085320999999999997 2024-09-18T00:58:26.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:26.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414955+0000", 2024-09-18T00:58:26.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3405999999999994e-05 2024-09-18T00:58:26.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.415023+0000", 2024-09-18T00:58:26.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7825999999999997e-05 2024-09-18T00:58:26.463 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.463 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.463 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.463 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15500 3.11 3:8ed6a97b:::rbd_data.117c53e529c1.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:26.463 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:26.463 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.638591142000003, 2024-09-18T00:58:26.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046818956000000002, 2024-09-18T00:58:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15500 2024-09-18T00:58:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:26.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:26.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814521+0000", 2024-09-18T00:58:26.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.977e-06 2024-09-18T00:58:26.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814536+0000", 2024-09-18T00:58:26.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.487e-05 2024-09-18T00:58:26.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814572+0000", 2024-09-18T00:58:26.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5567e-05 2024-09-18T00:58:26.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814588+0000", 2024-09-18T00:58:26.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6546e-05 2024-09-18T00:58:26.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814675+0000", 2024-09-18T00:58:26.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6675999999999995e-05 2024-09-18T00:58:26.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814839+0000", 2024-09-18T00:58:26.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016414199999999999 2024-09-18T00:58:26.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:26.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.819908+0000", 2024-09-18T00:58:26.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050695280000000002 2024-09-18T00:58:26.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:26.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861139+0000", 2024-09-18T00:58:26.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041230771999999999 2024-09-18T00:58:26.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:26.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861208+0000", 2024-09-18T00:58:26.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8722999999999999e-05 2024-09-18T00:58:26.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:26.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861277+0000", 2024-09-18T00:58:26.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9246999999999996e-05 2024-09-18T00:58:26.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861337+0000", 2024-09-18T00:58:26.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9908000000000003e-05 2024-09-18T00:58:26.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.471 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.472 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26860 3.a 3:523ff797:::rbd_data.117c4c7db1a0.0000000000000039:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:26.472 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:26.472 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.080417022999999, 2024-09-18T00:58:26.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028475341000000001, 2024-09-18T00:58:26.472 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.472 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:26.472 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.473 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.473 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.473 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26860 2024-09-18T00:58:26.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.473 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:26.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:26.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372693+0000", 2024-09-18T00:58:26.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0550000000000001e-06 2024-09-18T00:58:26.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372699+0000", 2024-09-18T00:58:26.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-18T00:58:26.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372714+0000", 2024-09-18T00:58:26.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4673e-05 2024-09-18T00:58:26.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372722+0000", 2024-09-18T00:58:26.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4659999999999999e-06 2024-09-18T00:58:26.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372787+0000", 2024-09-18T00:58:26.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5288000000000002e-05 2024-09-18T00:58:26.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372874+0000", 2024-09-18T00:58:26.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6485999999999998e-05 2024-09-18T00:58:26.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:26.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.377992+0000", 2024-09-18T00:58:26.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051177159999999996 2024-09-18T00:58:26.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:26.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401008+0000", 2024-09-18T00:58:26.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02301603 2024-09-18T00:58:26.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401026+0000", 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712e-05 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401082+0000", 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5648e-05 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401168+0000", 2024-09-18T00:58:26.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5476999999999995e-05 2024-09-18T00:58:26.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.480 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.480 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64289 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 3227648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.480 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:26.480 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86672732799999996, 2024-09-18T00:58:26.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025443502999999999, 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64289 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:26.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.84e-07 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586385+0000", 2024-09-18T00:58:26.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2060000000000001e-06 2024-09-18T00:58:26.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586392+0000", 2024-09-18T00:58:26.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:58:26.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586396+0000", 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5859999999999998e-06 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611580+0000", 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025183646000000001 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611667+0000", 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7350999999999998e-05 2024-09-18T00:58:26.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611825+0000", 2024-09-18T00:58:26.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015804999999999999 2024-09-18T00:58:26.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.485 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64403 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 475136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86578084499999997, 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025296714000000001, 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.486 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64403 2024-09-18T00:58:26.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.487 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:26.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:26.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587329+0000", 2024-09-18T00:58:26.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-18T00:58:26.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587331+0000", 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9889999999999999e-06 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587338+0000", 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5209999999999997e-06 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587343+0000", 2024-09-18T00:58:26.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:26.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612401+0000", 2024-09-18T00:58:26.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025057505000000001 2024-09-18T00:58:26.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612478+0000", 2024-09-18T00:58:26.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6817999999999995e-05 2024-09-18T00:58:26.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612625+0000", 2024-09-18T00:58:26.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014740099999999999 2024-09-18T00:58:26.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.491 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.492 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64404 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 655360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.492 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:26.492 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86575955800000004, 2024-09-18T00:58:26.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025540833999999998, 2024-09-18T00:58:26.492 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64404 2024-09-18T00:58:26.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.493 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:26.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587352+0000", 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587362+0000", 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0125e-05 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587367+0000", 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1150000000000002e-06 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612671+0000", 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025303638 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612751+0000", 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0107999999999999e-05 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612891+0000", 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013946 2024-09-18T00:58:26.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64405 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86573420300000004, 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025814888000000001, 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64405 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587379+0000", 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1439999999999999e-06 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587398+0000", 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587405+0000", 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3740000000000003e-06 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612931+0000", 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025526604000000001 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613008+0000", 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6508000000000007e-05 2024-09-18T00:58:26.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613190+0000", 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018205200000000001 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64406 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86568220299999998, 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026015033, 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64406 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:26.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587428+0000", 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0399999999999996e-07 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587431+0000", 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1870000000000001e-06 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587441+0000", 2024-09-18T00:58:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0349999999999999e-05 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587447+0000", 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.083e-06 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613236+0000", 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025788953 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613288+0000", 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2275999999999999e-05 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613442+0000", 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015358 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64407 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86565320400000001, 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026326130999999999, 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64407 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587459+0000", 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3750000000000001e-06 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587465+0000", 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7150000000000003e-06 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587469+0000", 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8060000000000002e-06 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613488+0000", 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026018645999999999 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613576+0000", 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7892000000000005e-05 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613782+0000", 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000206233 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.509 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64408 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86563296099999998, 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026568468000000001, 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64408 2024-09-18T00:58:26.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587477+0000", 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7500000000000001e-07 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587479+0000", 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1900000000000002e-06 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587487+0000", 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1869999999999997e-06 2024-09-18T00:58:26.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587492+0000", 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3149999999999999e-06 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613823+0000", 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026330339000000001 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613885+0000", 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2391999999999995e-05 2024-09-18T00:58:26.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614045+0000", 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015966999999999999 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64409 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.865609984, 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026785499000000001, 2024-09-18T00:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64409 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587500+0000", 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587502+0000", 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9420000000000002e-06 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587508+0000", 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1840000000000004e-06 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587513+0000", 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9640000000000002e-06 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614075+0000", 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026561889000000002 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614124+0000", 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9177e-05 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614285+0000", 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016103100000000001 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64410 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.518 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86559003499999998, 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026987559000000001, 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64410 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587520+0000", 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587522+0000", 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1500000000000002e-06 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587529+0000", 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5839999999999997e-06 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587534+0000", 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.826e-06 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614317+0000", 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026783065000000002 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614385+0000", 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7682999999999996e-05 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614507+0000", 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121859 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.522 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64411 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86556854999999999, 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027224245000000001, 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64411 2024-09-18T00:58:26.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587543+0000", 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587550+0000", 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5810000000000002e-06 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587555+0000", 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0490000000000003e-06 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614543+0000", 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026988056 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614619+0000", 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5932000000000004e-05 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614765+0000", 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014603899999999999 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64412 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2097152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86554780499999995, 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027478605, 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64412 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:26.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587562+0000", 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6499999999999999e-07 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587564+0000", 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3300000000000001e-06 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587571+0000", 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2949999999999999e-06 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587575+0000", 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6360000000000002e-06 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614798+0000", 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027222860000000002 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614874+0000", 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5383000000000007e-05 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615040+0000", 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016653600000000001 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64413 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86552700100000002, 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027709421000000001, 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64413 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:26.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587583+0000", 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587585+0000", 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587592+0000", 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9700000000000002e-06 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587597+0000", 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8289999999999998e-06 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615071+0000", 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027474218000000002 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615143+0000", 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1505000000000001e-05 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615292+0000", 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014900199999999999 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64414 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2457600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86550546299999997, 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027920637000000002, 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64414 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.08e-07 2024-09-18T00:58:26.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587606+0000", 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1299999999999999e-06 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587612+0000", 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1319999999999999e-06 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587618+0000", 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.147e-06 2024-09-18T00:58:26.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615323+0000", 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027705423 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615366+0000", 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2914000000000003e-05 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615524+0000", 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015848300000000001 2024-09-18T00:58:26.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64415 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86548114200000004, 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028241265000000002, 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64415 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587631+0000", 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5550000000000001e-06 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587639+0000", 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.807e-06 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587644+0000", 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2630000000000003e-06 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615633+0000", 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027989342 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615703+0000", 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9956000000000006e-05 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615869+0000", 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016597500000000001 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64416 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2818048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.865457845, 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028575605, 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64416 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587652+0000", 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587655+0000", 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587662+0000", 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7789999999999996e-06 2024-09-18T00:58:26.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587668+0000", 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2109999999999997e-06 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615916+0000", 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028248412000000001 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616009+0000", 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2682999999999993e-05 2024-09-18T00:58:26.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616227+0000", 2024-09-18T00:58:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218345 2024-09-18T00:58:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64417 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86543383399999996, 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028891495, 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64417 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587676+0000", 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.34e-07 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587683+0000", 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8739999999999997e-06 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587692+0000", 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2159999999999995e-06 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587698+0000", 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.716e-06 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616259+0000", 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028561733999999998 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616369+0000", 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110104 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616567+0000", 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000197417 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.552 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64418 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.86540338999999999, 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029401729000000001, 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64418 2024-09-18T00:58:26.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:26.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8400000000000004e-07 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587709+0000", 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5390000000000002e-06 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587717+0000", 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0819999999999999e-06 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587722+0000", 2024-09-18T00:58:26.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4140000000000002e-06 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616869+0000", 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029146763999999999 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616926+0000", 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7250000000000002e-05 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.617108+0000", 2024-09-18T00:58:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181096 2024-09-18T00:58:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:26.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:26.557 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:26.558 INFO:teuthology.orchestra.run.smithi042.stdout:2333 punch from 0x672e185 to 0x67326fd, (0x4578 bytes) 2024-09-18T00:58:26.558 INFO:teuthology.orchestra.run.smithi042.stdout:2334 punch from 0x22e17f4 to 0x22e6c19, (0x5425 bytes) 2024-09-18T00:58:26.558 INFO:teuthology.orchestra.run.smithi042.stdout:2335 trunc from 0x7a2fe4e to 0x67874bb 2024-09-18T00:58:26.559 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:26.560 INFO:teuthology.orchestra.run.smithi042.stdout:2336 punch from 0x457f78d to 0x4581d8d, (0x2600 bytes) 2024-09-18T00:58:26.560 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.560 INFO:teuthology.orchestra.run.smithi042.stdout:2341 write 0x5299cac thru 0x52a3b9f (0x9ef4 bytes) 2024-09-18T00:58:26.560 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.560 INFO:teuthology.orchestra.run.smithi042.stdout:2343 read 0x49ad094 thru 0x49b7581 (0xa4ee bytes) 2024-09-18T00:58:26.560 INFO:teuthology.orchestra.run.smithi042.stdout:2345 read 0x4b71666 thru 0x4b794b3 (0x7e4e bytes) 2024-09-18T00:58:26.560 INFO:teuthology.orchestra.run.smithi042.stdout:2346 writesame 0x456feba thru 0x4578839 data_size 0x58(0x8980 bytes) 2024-09-18T00:58:26.560 INFO:teuthology.orchestra.run.smithi042.stdout:2347 read 0x55035a3 thru 0x550677b (0x31d9 bytes) 2024-09-18T00:58:26.561 INFO:teuthology.orchestra.run.smithi042.stdout:2348 trunc from 0x67874bb to 0x3cb1be8 2024-09-18T00:58:26.561 INFO:teuthology.orchestra.run.smithi042.stdout:2349 writesame 0x92103f1 thru 0x9212b44 data_size 0x6(0x2754 bytes) 2024-09-18T00:58:26.561 INFO:teuthology.orchestra.run.smithi042.stdout:2351 writesame 0xd631b9 thru 0xd6ef69 data_size 0x1(0xbdb1 bytes) 2024-09-18T00:58:26.561 INFO:teuthology.orchestra.run.smithi042.stdout:2352 read 0x7968df4 thru 0x796cd32 (0x3f3f bytes) 2024-09-18T00:58:26.561 INFO:teuthology.orchestra.run.smithi042.stdout:2354 punch from 0x3f6193a to 0x3f70c7a, (0xf340 bytes) 2024-09-18T00:58:26.561 INFO:teuthology.orchestra.run.smithi042.stdout:2355 trunc from 0x9212b45 to 0xbf76c9 2024-09-18T00:58:26.593 INFO:teuthology.orchestra.run.smithi042.stdout:2357 write 0x6b2c3dd thru 0x6b2e21b (0x1e3f bytes) 2024-09-18T00:58:26.593 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:26.586+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:26.597 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.597 INFO:teuthology.orchestra.run.smithi042.stdout:2360 write 0x60a6a87 thru 0x60ad17c (0x66f6 bytes) 2024-09-18T00:58:26.603 INFO:teuthology.orchestra.run.smithi042.stdout:2361 trunc from 0x6b2e21c to 0x590e7b4 2024-09-18T00:58:26.611 INFO:teuthology.orchestra.run.smithi042.stdout:2362 write 0x2f18f81 thru 0x2f19bd0 (0xc50 bytes) 2024-09-18T00:58:26.615 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.615 INFO:teuthology.orchestra.run.smithi042.stdout:2365 read 0x7a5a61 thru 0x7b3c0d (0xe1ad bytes) 2024-09-18T00:58:26.628 INFO:teuthology.orchestra.run.smithi042.stdout:2367 write 0xa2e0f5f thru 0xa2e663e (0x56e0 bytes) 2024-09-18T00:58:26.632 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.637 INFO:teuthology.orchestra.run.smithi042.stdout:2371 trunc from 0xa2e663f to 0x5ecd51e 2024-09-18T00:58:26.654 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.654 INFO:teuthology.orchestra.run.smithi042.stdout:2373 writesame 0x18f85fc thru 0x18fb511 data_size 0x31(0x2f16 bytes) 2024-09-18T00:58:26.658 INFO:teuthology.orchestra.run.smithi042.stdout:2374 read 0x224d5c3 thru 0x2258945 (0xb383 bytes) 2024-09-18T00:58:26.658 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.658 INFO:teuthology.orchestra.run.smithi042.stdout:2376 read 0x15c5386 thru 0x15d1a5d (0xc6d8 bytes) 2024-09-18T00:58:26.663 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:26.658+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:26.678 INFO:teuthology.orchestra.run.smithi042.stdout:2377 writesame 0xc89acda thru 0xc8a8cdf data_size 0x9b(0xe006 bytes) 2024-09-18T00:58:26.679 INFO:teuthology.orchestra.run.smithi042.stdout:2379 read 0x1de80a thru 0x1e0b47 (0x233e bytes) 2024-09-18T00:58:26.679 INFO:teuthology.orchestra.run.smithi042.stdout:2380 write 0x9e02646 thru 0x9e119d0 (0xf38b bytes) 2024-09-18T00:58:26.693 INFO:teuthology.orchestra.run.smithi042.stdout:2381 trunc from 0xc8a8ce0 to 0xacd18c2 2024-09-18T00:58:26.706 INFO:teuthology.orchestra.run.smithi042.stdout:2382 trunc from 0xacd18c2 to 0x883b777 2024-09-18T00:58:26.710 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:26.710 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:26.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.710 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:65102 3.19 3.81f5e8b9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.710 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:26.710114+0000", 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.0049000000000001e-05, 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9442000000000001e-05, 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 65102 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:26.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710114+0000", 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710114+0000", 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710115+0000", 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1399999999999996e-07 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710119+0000", 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1649999999999999e-06 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710131+0000", 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1851999999999999e-05 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710136+0000", 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6439999999999997e-06 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710196+0000", 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9824999999999997e-05 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:65106 3.6 3:6615c3ea:::rbd_data.117c1b4dcc93.0000000000000054:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:26.710158+0000", 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.1530000000000002e-06, 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025655399999999998, 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:26.714 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 65106 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710158+0000", 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710158+0000", 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:26.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710158+0000", 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6699999999999999e-07 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710161+0000", 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3350000000000001e-06 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710170+0000", 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.85e-06 2024-09-18T00:58:26.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710174+0000", 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.527e-06 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710265+0000", 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0869e-05 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:26.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:26.710400+0000", 2024-09-18T00:58:26.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013541099999999999 2024-09-18T00:58:26.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:26.718 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:26.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:26.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:26.718 INFO:teuthology.orchestra.run.smithi195.stdout: ], 2024-09-18T00:58:26.718 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 2 2024-09-18T00:58:26.718 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:26.722 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.722 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.726 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:26.733 INFO:teuthology.orchestra.run.smithi042.stdout:2385 writesame 0xa63a940 thru 0xa641a7e data_size 0x1(0x713f bytes) 2024-09-18T00:58:26.733 INFO:teuthology.orchestra.run.smithi042.stdout:2387 read 0x896aaed thru 0x8977b56 (0xd06a bytes) 2024-09-18T00:58:26.733 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.733 INFO:teuthology.orchestra.run.smithi042.stdout:2391 write 0x6f7cb5b thru 0x6f8424a (0x76f0 bytes) 2024-09-18T00:58:26.744 INFO:teuthology.orchestra.run.smithi042.stdout:2392 trunc from 0xa641a7f to 0x5a5df08 2024-09-18T00:58:26.774 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:26.769+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:26.788 INFO:teuthology.orchestra.run.smithi042.stdout:2393 write 0xc7aa8e6 thru 0xc7abbf5 (0x1310 bytes) 2024-09-18T00:58:26.793 INFO:teuthology.orchestra.run.smithi042.stdout:2395 write 0x44d0e52 thru 0x44d55a9 (0x4758 bytes) 2024-09-18T00:58:26.798 INFO:teuthology.orchestra.run.smithi042.stdout:2396 read 0xc4d0a1c thru 0xc4d261d (0x1c02 bytes) 2024-09-18T00:58:26.799 INFO:teuthology.orchestra.run.smithi042.stdout:2398 write 0x54f23b2 thru 0x54fa6b1 (0x8300 bytes) 2024-09-18T00:58:26.813 INFO:teuthology.orchestra.run.smithi042.stdout:2400 trunc from 0xc7abbf6 to 0x60fca4e 2024-09-18T00:58:26.835 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.835 INFO:teuthology.orchestra.run.smithi042.stdout:2403 write 0x31a289f thru 0x31ad032 (0xa794 bytes) 2024-09-18T00:58:26.849 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:26.849 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:26.849 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:26.849 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:26.849 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:26.850 INFO:teuthology.orchestra.run.smithi042.stdout:2404 trunc from 0x60fca4e to 0x7da1fde 2024-09-18T00:58:26.863 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:26.863 INFO:teuthology.orchestra.run.smithi042.stdout:2405 read 0x5c3181c thru 0x5c35b72 (0x4357 bytes) 2024-09-18T00:58:26.864 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:26.858+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:26.883 INFO:teuthology.orchestra.run.smithi042.stdout:2408 writesame 0xaa85785 thru 0xaa88df2 data_size 0x1(0x366e bytes) 2024-09-18T00:58:26.883 INFO:teuthology.orchestra.run.smithi042.stdout:2409 read 0x6545d31 thru 0x6547946 (0x1c16 bytes) 2024-09-18T00:58:26.883 INFO:teuthology.orchestra.run.smithi042.stdout:2410 writesame 0x2fcf22 thru 0x308aec data_size 0x19(0xbbcb bytes) 2024-09-18T00:58:26.896 INFO:teuthology.orchestra.run.smithi042.stdout:2412 trunc from 0xaa88df3 to 0x5ef6e1b 2024-09-18T00:58:26.916 INFO:teuthology.orchestra.run.smithi042.stdout:2413 trunc from 0x5ef6e1b to 0x7e7731e 2024-09-18T00:58:26.928 INFO:teuthology.orchestra.run.smithi042.stdout:2415 punch from 0x2794723 to 0x2797250, (0x2b2d bytes) 2024-09-18T00:58:26.928 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:26.928 INFO:teuthology.orchestra.run.smithi042.stdout:2418 writesame 0x581e18b thru 0x5820532 data_size 0x1c(0x23a8 bytes) 2024-09-18T00:58:26.939 INFO:teuthology.orchestra.run.smithi042.stdout:2419 trunc from 0x7e7731e to 0x62b3f93 2024-09-18T00:58:26.965 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:26.961+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:26.966 INFO:teuthology.orchestra.run.smithi042.stdout:2420 trunc from 0x62b3f93 to 0xaa8d1a7 2024-09-18T00:58:26.979 INFO:teuthology.orchestra.run.smithi042.stdout:2421 writesame 0x255668e thru 0x2558252 data_size 0x1(0x1bc5 bytes) 2024-09-18T00:58:26.988 INFO:teuthology.orchestra.run.smithi042.stdout:2422 writesame 0x9a95b51 thru 0x9a97880 data_size 0x8(0x1d30 bytes) 2024-09-18T00:58:27.004 INFO:teuthology.orchestra.run.smithi042.stdout:2423 trunc from 0xaa8d1a7 to 0x75ba6ad 2024-09-18T00:58:27.030 INFO:teuthology.orchestra.run.smithi042.stdout:2424 write 0xc95e87 thru 0xc9bec4 (0x603e bytes) 2024-09-18T00:58:27.036 INFO:teuthology.orchestra.run.smithi042.stdout:2425 writesame 0x449ffe6 thru 0x44a72e2 data_size 0x1(0x72fd bytes) 2024-09-18T00:58:27.043 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1670 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1335296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.873455151999998, 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083399740999999999, 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1670 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166573+0000", 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1829999999999999e-06 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166578+0000", 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0030000000000002e-06 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166594+0000", 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6470999999999999e-05 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166607+0000", 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2831e-05 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249706+0000", 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083099022999999994 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249843+0000", 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136518 2024-09-18T00:58:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249971+0000", 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012871200000000001 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1671 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1376256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.873404375, 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083700574, 2024-09-18T00:58:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1671 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166623+0000", 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.066e-06 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166641+0000", 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7728e-05 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166659+0000", 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7850999999999999e-05 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166672+0000", 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3020000000000001e-05 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250034+0000", 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083362085000000002 2024-09-18T00:58:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250145+0000", 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111045 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250323+0000", 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017777900000000001 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1672 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.873330916, 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083853480999999994, 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1672 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166697+0000", 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0920000000000001e-06 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166702+0000", 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0799999999999996e-06 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166718+0000", 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6167999999999998e-05 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166745+0000", 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7177000000000001e-05 2024-09-18T00:58:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250382+0000", 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083636792000000001 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250476+0000", 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250549+0000", 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3282999999999996e-05 2024-09-18T00:58:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1673 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.873263807999997, 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084151880999999998, 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1673 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166764+0000", 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166770+0000", 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5969999999999999e-06 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166786+0000", 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6439999999999998e-05 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166799+0000", 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2934e-05 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250605+0000", 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083806357999999997 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250724+0000", 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011821500000000001 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250915+0000", 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000191251 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1674 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.873164223000003, 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084239101999999996, 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1674 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166864+0000", 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3629999999999999e-06 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166883+0000", 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9083e-05 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166902+0000", 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8711000000000001e-05 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166915+0000", 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3375e-05 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250962+0000", 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084047254000000002 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251029+0000", 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6315000000000005e-05 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251102+0000", 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3001000000000001e-05 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1675 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1540096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.873092372000002, 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084536891000000003, 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1675 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166936+0000", 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2720000000000001e-06 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166941+0000", 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7370000000000003e-06 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166958+0000", 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166972+0000", 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3569e-05 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251159+0000", 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084187467000000002 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251300+0000", 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014059200000000001 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251471+0000", 2024-09-18T00:58:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017168000000000001 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1676 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.873039491999997, 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084738637000000006, 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1676 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166988+0000", 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166993+0000", 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8339999999999998e-06 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167009+0000", 2024-09-18T00:58:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6028e-05 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167022+0000", 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2675e-05 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251517+0000", 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084494975999999999 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251576+0000", 2024-09-18T00:58:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8644999999999998e-05 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251726+0000", 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015039 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1677 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.872987821000002, 2024-09-18T00:58:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085055403000000002, 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1677 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167040+0000", 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167046+0000", 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167062+0000", 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6354e-05 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167075+0000", 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2965e-05 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251789+0000", 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084713940000000001 2024-09-18T00:58:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251916+0000", 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000127176 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252094+0000", 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017833599999999999 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1678 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.872936058000001, 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085316746999999998, 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1678 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167092+0000", 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167097+0000", 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7029999999999997e-06 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167113+0000", 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167125+0000", 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2517000000000001e-05 2024-09-18T00:58:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252157+0000", 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085031933000000004 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252277+0000", 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119708 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252407+0000", 2024-09-18T00:58:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013031800000000001 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1679 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.872885339, 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085557195000000003, 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1679 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167143+0000", 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167170+0000", 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7523000000000001e-05 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167189+0000", 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8530999999999999e-05 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167215+0000", 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6676999999999999e-05 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252445+0000", 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085229416000000002 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252529+0000", 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4740000000000005e-05 2024-09-18T00:58:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252699+0000", 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016920800000000001 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1680 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.872791882000001, 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085810110999999994, 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1680 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167236+0000", 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.048e-06 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167242+0000", 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6459999999999998e-06 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167259+0000", 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.696e-05 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167271+0000", 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.292e-05 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252839+0000", 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085567292000000003 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252924+0000", 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5511000000000002e-05 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253045+0000", 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120734 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1681 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.872685785999998, 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085887343000000005, 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1681 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167342+0000", 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167348+0000", 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4519999999999996e-06 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167372+0000", 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4261999999999999e-05 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167400+0000", 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-18T00:58:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253091+0000", 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085690895000000003 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253154+0000", 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3262000000000005e-05 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253228+0000", 2024-09-18T00:58:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4318000000000002e-05 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1682 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.872607705999997, 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.086061112999999995, 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1682 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168114+0000", 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00069532600000000002 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168123+0000", 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8319999999999995e-06 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168147+0000", 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3536999999999998e-05 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168163+0000", 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6317e-05 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253274+0000", 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085111390999999995 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253362+0000", 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7384000000000002e-05 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253480+0000", 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011832600000000001 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1683 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1867776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.871830574999997, 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085456638000000001, 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1683 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169938+0000", 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017419079999999999 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169951+0000", 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2534999999999999e-05 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169990+0000", 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9004999999999997e-05 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170011+0000", 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.156e-05 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253534+0000", 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083522759000000002 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253585+0000", 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1316999999999997e-05 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253653+0000", 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7553999999999997e-05 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1684 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.869980939000001, 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083787188999999998, 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1684 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170048+0000", 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.674e-06 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170055+0000", 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3549999999999999e-06 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170073+0000", 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7662000000000001e-05 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170087+0000", 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4229e-05 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253701+0000", 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083614346000000006 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253752+0000", 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0760000000000002e-05 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253833+0000", 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1162999999999995e-05 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1685 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.869920141999998, 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083900785000000005, 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.109 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1685 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170108+0000", 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.229e-06 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170114+0000", 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0079999999999999e-06 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170132+0000", 2024-09-18T00:58:27.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7734e-05 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170145+0000", 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2977e-05 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253866+0000", 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083721208000000005 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253934+0000", 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.834e-05 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254007+0000", 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3288999999999996e-05 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1686 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.869864931000002, 2024-09-18T00:58:27.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084083526000000006, 2024-09-18T00:58:27.114 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.114 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.114 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.114 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.114 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.114 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.114 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1686 2024-09-18T00:58:27.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.114 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170163+0000", 2024-09-18T00:58:27.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1909999999999999e-06 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170180+0000", 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7323e-05 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170198+0000", 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7723e-05 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170212+0000", 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3645999999999999e-05 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254050+0000", 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083838115000000005 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254136+0000", 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6106999999999994e-05 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254245+0000", 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010942099999999999 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1687 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.869796629, 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084186574, 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.118 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1687 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170647+0000", 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416649 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170655+0000", 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6510000000000007e-06 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170683+0000", 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7093999999999998e-05 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170701+0000", 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8383e-05 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254283+0000", 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083582091999999997 2024-09-18T00:58:27.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254339+0000", 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6187999999999998e-05 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254417+0000", 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7516999999999997e-05 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.122 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1688 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.869300914999997, 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083931026000000006, 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1688 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173553+0000", 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002827333 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173566+0000", 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3276e-05 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173603+0000", 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6177000000000003e-05 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173628+0000", 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5343000000000001e-05 2024-09-18T00:58:27.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254460+0000", 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.080831745999999996 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254547+0000", 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7045000000000004e-05 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254657+0000", 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011010600000000001 2024-09-18T00:58:27.126 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.265163713, 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.127 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.132 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:27.133 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:27.135 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:27.135 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:27.129+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:27.135 INFO:teuthology.orchestra.run.smithi042.stdout:2426 writesame 0xda6e809 thru 0xda77886 data_size 0xa(0x907e bytes) 2024-09-18T00:58:27.135 INFO:teuthology.orchestra.run.smithi042.stdout:2428 writesame 0x19e9285 thru 0x19ea149 data_size 0x1(0xec5 bytes) 2024-09-18T00:58:27.135 INFO:teuthology.orchestra.run.smithi042.stdout:2429 read 0xc0e053 thru 0xc0f091 (0x103f bytes) 2024-09-18T00:58:27.136 INFO:teuthology.orchestra.run.smithi042.stdout:2430 punch from 0x8d85ac9 to 0x8d9517d, (0xf6b4 bytes) 2024-09-18T00:58:27.136 INFO:teuthology.orchestra.run.smithi042.stdout:2431 punch from 0x35b5035 to 0x35bf0a0, (0xa06b bytes) 2024-09-18T00:58:27.136 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:27.136 INFO:teuthology.orchestra.run.smithi042.stdout:2436 write 0x72e0fdb thru 0x72eb394 (0xa3ba bytes) 2024-09-18T00:58:27.136 INFO:teuthology.orchestra.run.smithi042.stdout:2437 trunc from 0xda77887 to 0x5bfc72c 2024-09-18T00:58:27.157 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:27.157 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:27.177 INFO:teuthology.orchestra.run.smithi042.stdout:2440 writesame 0xb1db3d2 thru 0xb1e3215 data_size 0x2(0x7e44 bytes) 2024-09-18T00:58:27.177 INFO:teuthology.orchestra.run.smithi042.stdout:2441 punch from 0x2040261 to 0x204f36e, (0xf10d bytes) 2024-09-18T00:58:27.188 INFO:teuthology.orchestra.run.smithi042.stdout:2442 write 0xb40f29a thru 0xb41c1e8 (0xcf4f bytes) 2024-09-18T00:58:27.201 INFO:teuthology.orchestra.run.smithi042.stdout:2443 write 0x4ae1945 thru 0x4ae4469 (0x2b25 bytes) 2024-09-18T00:58:27.207 INFO:teuthology.orchestra.run.smithi042.stdout:2446 punch from 0x9bad875 to 0x9bb8065, (0xa7f0 bytes) 2024-09-18T00:58:27.217 INFO:teuthology.orchestra.run.smithi042.stdout:2448 write 0xb51af05 thru 0xb521cbe (0x6dba bytes) 2024-09-18T00:58:27.219 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:27.220 INFO:teuthology.orchestra.run.smithi042.stdout:2451 read 0xa4e5b41 thru 0xa4ee219 (0x86d9 bytes) 2024-09-18T00:58:27.220 INFO:teuthology.orchestra.run.smithi042.stdout:2452 writesame 0x82325cf thru 0x8233bf4 data_size 0xf(0x1626 bytes) 2024-09-18T00:58:27.226 INFO:teuthology.orchestra.run.smithi042.stdout:2453 trunc from 0xb521cbf to 0xc364ca3 2024-09-18T00:58:27.234 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:27.230+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:27.236 INFO:teuthology.orchestra.run.smithi042.stdout:2454 read 0x8405538 thru 0x840f579 (0xa042 bytes) 2024-09-18T00:58:27.236 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:58:27.252 INFO:teuthology.orchestra.run.smithi042.stdout:2457 writesame 0xd8a23cd thru 0xd8a4010 data_size 0x3(0x1c44 bytes) 2024-09-18T00:58:27.252 INFO:teuthology.orchestra.run.smithi042.stdout:2459 read 0xc12220f thru 0xc1272bd (0x50af bytes) 2024-09-18T00:58:27.253 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:27.254 INFO:teuthology.orchestra.run.smithi042.stdout:2461 trunc from 0xd8a4011 to 0xc3da93b 2024-09-18T00:58:27.267 INFO:teuthology.orchestra.run.smithi042.stdout:2462 writesame 0x9bd39bb thru 0x9bda77b data_size 0x1(0x6dc1 bytes) 2024-09-18T00:58:27.296 INFO:teuthology.orchestra.run.smithi042.stdout:2465 writesame 0x7864f43 thru 0x786769c data_size 0x2e(0x275a bytes) 2024-09-18T00:58:27.298 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:27.298 INFO:teuthology.orchestra.run.smithi042.stdout:2468 read 0xe07754 thru 0xe0cbc4 (0x5471 bytes) 2024-09-18T00:58:27.299 INFO:teuthology.orchestra.run.smithi042.stdout:2469 punch from 0x2babdfb to 0x2bace0b, (0x1010 bytes) 2024-09-18T00:58:27.317 INFO:teuthology.orchestra.run.smithi042.stdout:2471 trunc from 0xc3da93b to 0xbd2c05 2024-09-18T00:58:27.320 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:27.320 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:27.324 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:27.324 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:65421 3.18 3:1b8be77f:::rbd_data.117c1b4dcc93.0000000000000016:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:27.320015+0000", 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.0036820490000000002, 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038111949999999999, 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "waiting for sub ops", 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 65421 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.325 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.320015+0000", 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.320015+0000", 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.320018+0000", 2024-09-18T00:58:27.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.32e-06 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.320032+0000", 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3736e-05 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.320076+0000", 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3705999999999998e-05 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.320100+0000", 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4236999999999999e-05 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.320265+0000", 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000164489 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.320473+0000", 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020842500000000001 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.320777+0000", 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000304371 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.322398+0000", 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001621054 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.329 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:65431 3.1e 3:79f77328:::rbd_data.117c1b4dcc93.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:27.322807+0000", 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.00089060099999999998, 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013590340000000001, 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "waiting for sub ops", 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 65431 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.322807+0000", 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.322807+0000", 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.322808+0000", 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6200000000000006e-07 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.322811+0000", 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8249999999999998e-06 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.322821+0000", 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0101e-05 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.322828+0000", 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8979999999999999e-06 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.322975+0000", 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146277 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.323219+0000", 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024410499999999999 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.323480+0000", 2024-09-18T00:58:27.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026106300000000002 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:65434 3.10 e64/24 3:0f99b081:::rbd_data.117c1b4dcc93.0000000000000032:head v 64'216, mlcod=63'214)", 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:27.323772+0000", 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4294967295.9999251, 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00060274600000000003, 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.323772+0000", 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.323772+0000", 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.323773+0000", 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.44e-07 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.323779+0000", 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0299999999999999e-06 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.323788+0000", 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7439999999999996e-06 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.323796+0000", 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8249999999999994e-06 2024-09-18T00:58:27.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.323868+0000", 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1928999999999995e-05 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.323888+0000", 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9752e-05 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(client.4476.0:65421 3.18 e64/24)", 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:27.324458+0000", 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4294967295.9992394, 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1989999999999993e-05, 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "queued for pg", 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.324458+0000", 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.324458+0000", 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.324459+0000", 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9269999999999999e-06 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.324471+0000", 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1853e-05 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.324498+0000", 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6414000000000002e-05 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:27.324515+0000", 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7329999999999998e-05 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: ], 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 4 2024-09-18T00:58:27.340 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:27.341 INFO:teuthology.orchestra.run.smithi042.stdout:2474 read 0x1205f1 thru 0x127584 (0x6f94 bytes) 2024-09-18T00:58:27.341 INFO:teuthology.orchestra.run.smithi042.stdout:2476 punch from 0x1c4bde to 0x1c7354, (0x2776 bytes) 2024-09-18T00:58:27.342 INFO:teuthology.orchestra.run.smithi042.stdout:2477 read 0x708cd4 thru 0x710d72 (0x809f bytes) 2024-09-18T00:58:27.342 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:27.337+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:27.343 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:27.345 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:27.345 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:27.359 INFO:teuthology.orchestra.run.smithi042.stdout:2480 writesame 0xace92d0 thru 0xacf7533 data_size 0x2(0xe264 bytes) 2024-09-18T00:58:27.359 INFO:teuthology.orchestra.run.smithi042.stdout:2481 read 0x2593138 thru 0x259c190 (0x9059 bytes) 2024-09-18T00:58:27.360 INFO:teuthology.orchestra.run.smithi042.stdout:2482 write 0x351d72e thru 0x3521757 (0x402a bytes) 2024-09-18T00:58:27.363 INFO:teuthology.orchestra.run.smithi042.stdout:2483 write 0x2db18f9 thru 0x2db6f27 (0x562f bytes) 2024-09-18T00:58:27.364 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:27.364 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:27.373 INFO:teuthology.orchestra.run.smithi042.stdout:2484 writesame 0xb02ab6d thru 0xb0334ab data_size 0x1(0x893f bytes) 2024-09-18T00:58:27.373 INFO:teuthology.orchestra.run.smithi042.stdout:2485 write 0x74ba5a6 thru 0x74c995a (0xf3b5 bytes) 2024-09-18T00:58:27.380 INFO:teuthology.orchestra.run.smithi042.stdout:2489 write 0x3468cd6 thru 0x34716bf (0x89ea bytes) 2024-09-18T00:58:27.386 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:27.390 INFO:teuthology.orchestra.run.smithi042.stdout:2491 write 0xd54e72f thru 0xd55833b (0x9c0d bytes) 2024-09-18T00:58:27.398 INFO:teuthology.orchestra.run.smithi042.stdout:2492 punch from 0xc6d70d2 to 0xc6e4197, (0xd0c5 bytes) 2024-09-18T00:58:27.398 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:27.398 INFO:teuthology.orchestra.run.smithi042.stdout:2494 punch from 0x243d477 to 0x2446ae5, (0x966e bytes) 2024-09-18T00:58:27.398 INFO:teuthology.orchestra.run.smithi042.stdout:2495 read 0x48e64c9 thru 0x48f5321 (0xee59 bytes) 2024-09-18T00:58:27.398 INFO:teuthology.orchestra.run.smithi042.stdout:2496 read 0x2598e01 thru 0x259fd63 (0x6f63 bytes) 2024-09-18T00:58:27.398 INFO:teuthology.orchestra.run.smithi042.stdout:2497 writesame 0x2d3bfeb thru 0x2d49191 data_size 0x1(0xd1a7 bytes) 2024-09-18T00:58:27.407 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:27.414 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:27.414 INFO:teuthology.orchestra.run.smithi042.stdout:2499 punch from 0x34b3f7e to 0x34ba49b, (0x651d bytes) 2024-09-18T00:58:27.415 INFO:teuthology.orchestra.run.smithi042.stdout:2500 writesame 0x90b85f2 thru 0x90c7235 data_size 0x2(0xec44 bytes) 2024-09-18T00:58:27.425 INFO:teuthology.orchestra.run.smithi042.stdout:2504 write 0x6a0defd thru 0x6a146a7 (0x67ab bytes) 2024-09-18T00:58:27.430 INFO:teuthology.orchestra.run.smithi042.stdout:2505 clone 22 order 21 su 32768 sc 5 2024-09-18T00:58:27.431 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:27.431 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:27.442 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:27.437+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:27.449 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:27.449 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:27.467 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:27.467 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:27.519 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:27.519 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:27.533 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:27.544 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:27.538+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:27.645 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:27.642+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:27.699 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:27.700 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:27.714 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:27.761 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:27.757+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:27.835 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:27.909 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:27.912 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:27.912 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:27.905+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:28.012 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:28.006+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:28.084 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:28.085 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:28.099 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:28.145 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:28.141+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:28.214 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:28.210+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:28.216 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:28.216 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:28.230 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:28.317 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:28.313+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:28.406 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:28.406 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone21 from 0xd55833c (overlap 0x33bd93) to 0x52c8e 2024-09-18T00:58:28.415 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:28.410+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:28.420 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:28.516 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:28.510+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:28.582 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:28.582 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:28.597 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:28.604 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-18T00:58:28.641 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:28.637+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:28.717 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:28.717 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:28.718 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:28.714+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:28.727 INFO:teuthology.orchestra.run.smithi042.stdout:2506 punch from 0xa50c7d0 to 0xa515b24, (0x9354 bytes) 2024-09-18T00:58:28.731 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:28.743 INFO:teuthology.orchestra.run.smithi042.stdout:2511 trunc from 0xd55833c to 0xa8dcae4 2024-09-18T00:58:28.778 INFO:teuthology.orchestra.run.smithi042.stdout:2514 write 0xbf809ce thru 0xbf8d73f (0xcd72 bytes) 2024-09-18T00:58:28.783 INFO:teuthology.orchestra.run.smithi042.stdout:2515 write 0xa7a86ac thru 0xa7b7109 (0xea5e bytes) 2024-09-18T00:58:28.786 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:28.786 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:28.786 INFO:teuthology.orchestra.run.smithi042.stdout:2520 write 0xad01293 thru 0xad0a38c (0x90fa bytes) 2024-09-18T00:58:28.794 INFO:teuthology.orchestra.run.smithi042.stdout:2523 write 0x1d77d52 thru 0x1d844d0 (0xc77f bytes) 2024-09-18T00:58:28.805 INFO:teuthology.orchestra.run.smithi042.stdout:2524 read 0x5ca0e45 thru 0x5ca9419 (0x85d5 bytes) 2024-09-18T00:58:28.807 INFO:teuthology.orchestra.run.smithi042.stdout:2525 read 0x7863710 thru 0x7865aba (0x23ab bytes) 2024-09-18T00:58:28.809 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:28.809 INFO:teuthology.orchestra.run.smithi042.stdout:2528 punch from 0x97ad2e to 0x98a550, (0xf822 bytes) 2024-09-18T00:58:28.809 INFO:teuthology.orchestra.run.smithi042.stdout:2529 punch from 0x297599b to 0x297c7b0, (0x6e15 bytes) 2024-09-18T00:58:28.809 INFO:teuthology.orchestra.run.smithi042.stdout:2530 read 0xadb8606 thru 0xadc2e7f (0xa87a bytes) 2024-09-18T00:58:28.809 INFO:teuthology.orchestra.run.smithi042.stdout:2531 read 0x39e8098 thru 0x39ec01d (0x3f86 bytes) 2024-09-18T00:58:28.819 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:28.813+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:28.835 INFO:teuthology.orchestra.run.smithi042.stdout:2532 trunc from 0xbf8d740 to 0x23e4f3c 2024-09-18T00:58:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:2533 writesame 0x83d84a thru 0x843941 data_size 0x3a(0x60f8 bytes) 2024-09-18T00:58:28.882 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:28.883 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:28.905 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:28.912 INFO:teuthology.orchestra.run.smithi042.stdout:2534 read 0x1bea393 thru 0x1bf7743 (0xd3b1 bytes) 2024-09-18T00:58:28.919 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:28.913+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:28.925 INFO:teuthology.orchestra.run.smithi042.stdout:2535 write 0x2b58051 thru 0x2b60c1b (0x8bcb bytes) 2024-09-18T00:58:28.935 INFO:teuthology.orchestra.run.smithi042.stdout:2536 read 0x24c3bf5 thru 0x24c9854 (0x5c60 bytes) 2024-09-18T00:58:28.935 INFO:teuthology.orchestra.run.smithi042.stdout:2537 read 0x117ed61 thru 0x118b8f0 (0xcb90 bytes) 2024-09-18T00:58:28.955 INFO:teuthology.orchestra.run.smithi042.stdout:2540 trunc from 0x2b60c1c to 0xc304ef8 2024-09-18T00:58:28.975 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:28.977 INFO:teuthology.orchestra.run.smithi042.stdout:2542 trunc from 0xc304ef8 to 0xae8729b 2024-09-18T00:58:28.995 INFO:teuthology.orchestra.run.smithi042.stdout:2544 trunc from 0xae8729b to 0x651092e 2024-09-18T00:58:29.007 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:29.021 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:29.013+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:29.033 INFO:teuthology.orchestra.run.smithi042.stdout:2549 write 0xe74f28e thru 0xe756e10 (0x7b83 bytes) 2024-09-18T00:58:29.041 INFO:teuthology.orchestra.run.smithi042.stdout:2550 read 0xd36d8 thru 0xd3d59 (0x682 bytes) 2024-09-18T00:58:29.043 INFO:teuthology.orchestra.run.smithi042.stdout:2551 read 0x9c58ff8 thru 0x9c66ad7 (0xdae0 bytes) 2024-09-18T00:58:29.043 INFO:teuthology.orchestra.run.smithi042.stdout:2552 write 0x8c8099b thru 0x8c84e20 (0x4486 bytes) 2024-09-18T00:58:29.054 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:29.054 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:29.054 INFO:teuthology.orchestra.run.smithi042.stdout:2554 writesame 0x98b79a6 thru 0x98c2671 data_size 0x2(0xaccc bytes) 2024-09-18T00:58:29.069 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:29.084 INFO:teuthology.orchestra.run.smithi042.stdout:2555 punch from 0xcc6f5de to 0xcc71ef9, (0x291b bytes) 2024-09-18T00:58:29.085 INFO:teuthology.orchestra.run.smithi042.stdout:2556 punch from 0x4fca80b to 0x4fd4450, (0x9c45 bytes) 2024-09-18T00:58:29.085 INFO:teuthology.orchestra.run.smithi042.stdout:2558 writesame 0xb4bf52 thru 0xb54a89 data_size 0x87(0x8b38 bytes) 2024-09-18T00:58:29.097 INFO:teuthology.orchestra.run.smithi042.stdout:2559 write 0xce9238a thru 0xce96218 (0x3e8f bytes) 2024-09-18T00:58:29.104 INFO:teuthology.orchestra.run.smithi042.stdout:2561 read 0x3da4fe7 thru 0x3da8b2c (0x3b46 bytes) 2024-09-18T00:58:29.104 INFO:teuthology.orchestra.run.smithi042.stdout:2563 write 0xb6b81fe thru 0xb6c1952 (0x9755 bytes) 2024-09-18T00:58:29.111 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:29.111 INFO:teuthology.orchestra.run.smithi042.stdout:2567 punch from 0xae177a1 to 0xae18186, (0x9e5 bytes) 2024-09-18T00:58:29.111 INFO:teuthology.orchestra.run.smithi042.stdout:2568 write 0xe1dd025 thru 0xe1e48d8 (0x78b4 bytes) 2024-09-18T00:58:29.118 INFO:teuthology.orchestra.run.smithi042.stdout:2570 write 0xd1ae872 thru 0xd1b6aa6 (0x8235 bytes) 2024-09-18T00:58:29.122 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:29.117+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:2571 write 0x149d965 thru 0x14ab947 (0xdfe3 bytes) 2024-09-18T00:58:29.135 INFO:teuthology.orchestra.run.smithi042.stdout:2572 write 0x65984a6 thru 0x65a1eb7 (0x9a12 bytes) 2024-09-18T00:58:29.159 INFO:teuthology.orchestra.run.smithi042.stdout:2573 trunc from 0xe756e11 to 0x5672cf3 2024-09-18T00:58:29.184 INFO:teuthology.orchestra.run.smithi042.stdout:2574 punch from 0x2ae59d9 to 0x2af3e08, (0xe42f bytes) 2024-09-18T00:58:29.184 INFO:teuthology.orchestra.run.smithi042.stdout:2575 punch from 0x4e6fa28 to 0x4e78c28, (0x9200 bytes) 2024-09-18T00:58:29.184 INFO:teuthology.orchestra.run.smithi042.stdout:2576 punch from 0x50e1b87 to 0x50e92b4, (0x772d bytes) 2024-09-18T00:58:29.185 INFO:teuthology.orchestra.run.smithi042.stdout:2577 write 0x13e98f8 thru 0x13f6a61 (0xd16a bytes) 2024-09-18T00:58:29.199 INFO:teuthology.orchestra.run.smithi042.stdout:2581 read 0x309d265 thru 0x30a144d (0x41e9 bytes) 2024-09-18T00:58:29.199 INFO:teuthology.orchestra.run.smithi042.stdout:2582 read 0xc5a49c thru 0xc61f16 (0x7a7b bytes) 2024-09-18T00:58:29.221 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:29.221 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:29.222 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:29.218+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:29.223 INFO:teuthology.orchestra.run.smithi042.stdout:2583 write 0xc4876f1 thru 0xc492808 (0xb118 bytes) 2024-09-18T00:58:29.234 INFO:teuthology.orchestra.run.smithi042.stdout:2586 writesame 0xbb9f29f thru 0xbbac099 data_size 0xbd(0xcdfb bytes) 2024-09-18T00:58:29.237 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:29.247 INFO:teuthology.orchestra.run.smithi042.stdout:2590 punch from 0xb4cd45a to 0xb4d20e8, (0x4c8e bytes) 2024-09-18T00:58:29.247 INFO:teuthology.orchestra.run.smithi042.stdout:2591 read 0x5c55a5f thru 0x5c65a38 (0xffda bytes) 2024-09-18T00:58:29.247 INFO:teuthology.orchestra.run.smithi042.stdout:2592 read 0x99657af thru 0x9972065 (0xc8b7 bytes) 2024-09-18T00:58:29.247 INFO:teuthology.orchestra.run.smithi042.stdout:2593 write 0x99a4b75 thru 0x99b2486 (0xd912 bytes) 2024-09-18T00:58:29.258 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:29.258 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:29.271 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:58:29.273 INFO:teuthology.orchestra.run.smithi042.stdout:2596 writesame 0xcd33097 thru 0xcd34fc0 data_size 0x1(0x1f2a bytes) 2024-09-18T00:58:29.273 INFO:teuthology.orchestra.run.smithi042.stdout:2597 read 0x3b65980 thru 0x3b6cae0 (0x7161 bytes) 2024-09-18T00:58:29.273 INFO:teuthology.orchestra.run.smithi042.stdout:2598 punch from 0x12d14b3 to 0x12df8f3, (0xe440 bytes) 2024-09-18T00:58:29.273 INFO:teuthology.orchestra.run.smithi042.stdout:2599 write 0x383e8cb thru 0x384022d (0x1963 bytes) 2024-09-18T00:58:29.281 INFO:teuthology.orchestra.run.smithi042.stdout:2601 punch from 0x5396e18 to 0x53a3563, (0xc74b bytes) 2024-09-18T00:58:29.281 INFO:teuthology.orchestra.run.smithi042.stdout:2604 write 0x4463bf5 thru 0x446ec84 (0xb090 bytes) 2024-09-18T00:58:29.290 INFO:teuthology.orchestra.run.smithi042.stdout:2605 write 0x8afb3c8 thru 0x8affcbe (0x48f7 bytes) 2024-09-18T00:58:29.295 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:29.296 INFO:teuthology.orchestra.run.smithi042.stdout:2608 write 0xc82b485 thru 0xc82f624 (0x41a0 bytes) 2024-09-18T00:58:29.302 INFO:teuthology.orchestra.run.smithi042.stdout:2609 punch from 0x2fb06c to 0x2fcd66, (0x1cfa bytes) 2024-09-18T00:58:29.302 INFO:teuthology.orchestra.run.smithi042.stdout:2610 read 0x3e07eed thru 0x3e0aaad (0x2bc1 bytes) 2024-09-18T00:58:29.308 INFO:teuthology.orchestra.run.smithi042.stdout:2611 trunc from 0xcd34fc1 to 0x8d3d8e0 2024-09-18T00:58:29.324 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:29.317+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:2612 write 0x3a2d79c thru 0x3a34cef (0x7554 bytes) 2024-09-18T00:58:29.342 INFO:teuthology.orchestra.run.smithi042.stdout:2613 clone 23 order 24 su 131072 sc 7 2024-09-18T00:58:29.381 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:29.382 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:29.396 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:29.425 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:29.421+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:29.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:29.526 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:29.521+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:29.568 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:29.568 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:29.582 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:29.628 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:29.628 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:29.622+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:29.645 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:29.663 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:29.680 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:29.701 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:29.701 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:29.703 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 66.412631626999996, 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:58:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:58:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:58:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:58:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:58:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:58:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:58:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7423 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.216333077000002, 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045296661000000002, 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7423 2024-09-18T00:58:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485619+0000", 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485621+0000", 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485627+0000", 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.156e-06 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485633+0000", 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1789999999999999e-06 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530788+0000", 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045155001 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530853+0000", 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5634000000000002e-05 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530915+0000", 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2022999999999999e-05 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7424 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1146880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.216312086999999, 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045419355000000002, 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7424 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485640+0000", 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485642+0000", 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0640000000000001e-06 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485649+0000", 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485654+0000", 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2329999999999997e-06 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.530941+0000", 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045286602000000002 2024-09-18T00:58:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531004+0000", 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3455999999999997e-05 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531059+0000", 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4771000000000001e-05 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.216290573999999, 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:58:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:58:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:58:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:58:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:58:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:58:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.216269136999998, 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:58:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:58:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.216173810000001, 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:58:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.216148371000003, 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.216125847999997, 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:58:29.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:58:29.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.216102407999998, 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:58:29.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:29.746 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.216078598000003, 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:29.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:58:29.750 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.216056233000003, 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:58:29.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:58:29.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.216029616, 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.755 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.759 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.211877844999997, 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:58:29.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:58:29.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.211857102000003, 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:58:29.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.768 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.211811537000003, 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:58:29.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.211785196000001, 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:58:29.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:58:29.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.211762526000001, 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.778 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:58:29.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:29.782 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.211744152999998, 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:29.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.211724365000002, 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:58:29.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:58:29.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:58:29.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.211697495000003, 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:29.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:29.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:58:29.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:58:29.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:58:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:58:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:58:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:58:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:58:29.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:29.796 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:29.796 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:29.797 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:29.798 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:29.798 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:29.798 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:29.798 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:29.799 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:29.799 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:29.800 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:29.790+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:29.898 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:29.894+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:29.966 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:29.966 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:29.966 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:29.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:30.022 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:30.018+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:30.102 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:30.098+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:30.146 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:30.146 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:30.147 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:30.161 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:30.202 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:30.198+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:30.260 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:58:30.306 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:30.302+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.267488667000002, 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.347 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:30.351 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.266559076, 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:30.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:58:30.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.266545016000002, 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.356 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:58:30.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.265254441000003, 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:58:30.360 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:58:30.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.265233969000001, 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:58:30.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:58:30.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:58:30.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.262439516000001, 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.369 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:58:30.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:58:30.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.262424332000002, 2024-09-18T00:58:30.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:30.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:58:30.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.262401386000001, 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:58:30.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.382 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.258223747000002, 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:58:30.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.258202529999998, 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.387 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.391 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.258181147000002, 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:58:30.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:58:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.254281292999998, 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:58:30.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:58:30.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.400 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.254267091999999, 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:58:30.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:58:30.404 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.254249055000002, 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.405 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:58:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.254234988999997, 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:58:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:58:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.250209638000001, 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:58:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.250195626, 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.250181662000003, 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:58:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.488323708000003, 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:58:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.362847337000002, 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:58:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:58:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.438 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:30.439 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:30.439 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone22 from 0x8d3d8e0 (overlap 0x23e4f3c) to 0x0 2024-09-18T00:58:30.440 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:30.434+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:30.540 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:30.533+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:30.620 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:30.621 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:30.621 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:30.621 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:30.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:30.641 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:30.637+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:30.645 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-18T00:58:30.664 INFO:teuthology.orchestra.run.smithi042.stdout:2615 punch from 0x5c2e3f2 to 0x5c399ef, (0xb5fd bytes) 2024-09-18T00:58:30.670 INFO:teuthology.orchestra.run.smithi042.stdout:2616 punch from 0x67a4752 to 0x67b0ad8, (0xc386 bytes) 2024-09-18T00:58:30.670 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:30.670 INFO:teuthology.orchestra.run.smithi042.stdout:2620 read 0x82a526e thru 0x82a6876 (0x1609 bytes) 2024-09-18T00:58:30.670 INFO:teuthology.orchestra.run.smithi042.stdout:2622 writesame 0x85cc276 thru 0x85d3fb3 data_size 0x11(0x7d3e bytes) 2024-09-18T00:58:30.681 INFO:teuthology.orchestra.run.smithi042.stdout:2623 writesame 0x3e8ffa4 thru 0x3e9770f data_size 0x2(0x776c bytes) 2024-09-18T00:58:30.741 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:30.738+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:30.744 INFO:teuthology.orchestra.run.smithi042.stdout:2625 read 0x8804832 thru 0x8809cc9 (0x5498 bytes) 2024-09-18T00:58:30.745 INFO:teuthology.orchestra.run.smithi042.stdout:2626 write 0x3deb846 thru 0x3dee2a3 (0x2a5e bytes) 2024-09-18T00:58:30.783 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:30.783 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:30.783 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:30.800 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:30.817 INFO:teuthology.orchestra.run.smithi042.stdout:2627 punch from 0x78ebe76 to 0x78fa359, (0xe4e3 bytes) 2024-09-18T00:58:30.817 INFO:teuthology.orchestra.run.smithi042.stdout:2628 read 0x4dd5fbe thru 0x4de0fce (0xb011 bytes) 2024-09-18T00:58:30.831 INFO:teuthology.orchestra.run.smithi042.stdout:2630 read 0x461529e thru 0x461bf47 (0x6caa bytes) 2024-09-18T00:58:30.833 INFO:teuthology.orchestra.run.smithi042.stdout:2631 punch from 0x6e06b3 to 0x6e1ce2, (0x162f bytes) 2024-09-18T00:58:30.833 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:30.833 INFO:teuthology.orchestra.run.smithi042.stdout:2634 write 0x4796b4c thru 0x479e633 (0x7ae8 bytes) 2024-09-18T00:58:30.843 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:30.838+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:30.846 INFO:teuthology.orchestra.run.smithi042.stdout:2635 trunc from 0x8d3d8e0 to 0x795209f 2024-09-18T00:58:30.916 INFO:teuthology.orchestra.run.smithi042.stdout:2637 punch from 0x2403168 to 0x24075a8, (0x4440 bytes) 2024-09-18T00:58:30.916 INFO:teuthology.orchestra.run.smithi042.stdout:2640 write 0x65c4155 thru 0x65c4709 (0x5b5 bytes) 2024-09-18T00:58:30.943 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:30.937+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:30.972 INFO:teuthology.orchestra.run.smithi042.stdout:2641 write 0x8766301 thru 0x87736b7 (0xd3b7 bytes) 2024-09-18T00:58:30.977 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:30.977 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:30.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:30.977 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:30.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.977 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:58:30.977 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:30.977 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 63.660205177000002, 2024-09-18T00:58:30.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101264911, 2024-09-18T00:58:30.977 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:30.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4226", 2024-09-18T00:58:30.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2155402750", 2024-09-18T00:58:30.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:30.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.978 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313760+0000", 2024-09-18T00:58:30.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.612e-06 2024-09-18T00:58:30.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313768+0000", 2024-09-18T00:58:30.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9249999999999995e-06 2024-09-18T00:58:30.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313786+0000", 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8253999999999998e-05 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313803+0000", 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313935+0000", 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013180399999999999 2024-09-18T00:58:30.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:30.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313954+0000", 2024-09-18T00:58:30.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8768999999999999e-05 2024-09-18T00:58:30.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.411874+0000", 2024-09-18T00:58:30.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.097920646 2024-09-18T00:58:30.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412292+0000", 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041777699999999998 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412675+0000", 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038262300000000001 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:30.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414019+0000", 2024-09-18T00:58:30.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013436699999999999 2024-09-18T00:58:30.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:30.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414872+0000", 2024-09-18T00:58:30.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085320999999999997 2024-09-18T00:58:30.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:30.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414955+0000", 2024-09-18T00:58:30.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3405999999999994e-05 2024-09-18T00:58:30.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.415023+0000", 2024-09-18T00:58:30.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7825999999999997e-05 2024-09-18T00:58:30.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.985 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.985 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15500 3.11 3:8ed6a97b:::rbd_data.117c53e529c1.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:30.986 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:30.986 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.159445157, 2024-09-18T00:58:30.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046818956000000002, 2024-09-18T00:58:30.986 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.986 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:30.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15500 2024-09-18T00:58:30.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814521+0000", 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.977e-06 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814536+0000", 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.487e-05 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814572+0000", 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5567e-05 2024-09-18T00:58:30.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814588+0000", 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6546e-05 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814675+0000", 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6675999999999995e-05 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814839+0000", 2024-09-18T00:58:30.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016414199999999999 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.819908+0000", 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050695280000000002 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861139+0000", 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041230771999999999 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861208+0000", 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8722999999999999e-05 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861277+0000", 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9246999999999996e-05 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861337+0000", 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9908000000000003e-05 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.991 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26860 3.a 3:523ff797:::rbd_data.117c4c7db1a0.0000000000000039:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.601271038, 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028475341000000001, 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.992 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:30.993 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26860 2024-09-18T00:58:30.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.993 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:30.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:30.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:30.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372693+0000", 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0550000000000001e-06 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:30.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372699+0000", 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372714+0000", 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4673e-05 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372722+0000", 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4659999999999999e-06 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372787+0000", 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5288000000000002e-05 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372874+0000", 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6485999999999998e-05 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.377992+0000", 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051177159999999996 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401008+0000", 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02301603 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401026+0000", 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712e-05 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401082+0000", 2024-09-18T00:58:30.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5648e-05 2024-09-18T00:58:30.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:30.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401168+0000", 2024-09-18T00:58:30.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5476999999999995e-05 2024-09-18T00:58:30.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.998 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:30.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64289 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 3227648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3875813429999999, 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025443502999999999, 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:30.999 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64289 2024-09-18T00:58:31.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.000 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:31.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.84e-07 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586385+0000", 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2060000000000001e-06 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586392+0000", 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586396+0000", 2024-09-18T00:58:31.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5859999999999998e-06 2024-09-18T00:58:31.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611580+0000", 2024-09-18T00:58:31.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025183646000000001 2024-09-18T00:58:31.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611667+0000", 2024-09-18T00:58:31.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7350999999999998e-05 2024-09-18T00:58:31.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611825+0000", 2024-09-18T00:58:31.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015804999999999999 2024-09-18T00:58:31.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.004 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64403 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 475136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.38663486, 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025296714000000001, 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64403 2024-09-18T00:58:31.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.006 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:31.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:31.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587329+0000", 2024-09-18T00:58:31.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-18T00:58:31.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587331+0000", 2024-09-18T00:58:31.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9889999999999999e-06 2024-09-18T00:58:31.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587338+0000", 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5209999999999997e-06 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587343+0000", 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612401+0000", 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025057505000000001 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612478+0000", 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6817999999999995e-05 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612625+0000", 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014740099999999999 2024-09-18T00:58:31.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.010 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.010 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.010 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64404 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 655360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.010 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:31.010 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.386613573, 2024-09-18T00:58:31.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025540833999999998, 2024-09-18T00:58:31.010 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64404 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:31.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:31.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:31.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587352+0000", 2024-09-18T00:58:31.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-18T00:58:31.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587362+0000", 2024-09-18T00:58:31.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0125e-05 2024-09-18T00:58:31.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587367+0000", 2024-09-18T00:58:31.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1150000000000002e-06 2024-09-18T00:58:31.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612671+0000", 2024-09-18T00:58:31.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025303638 2024-09-18T00:58:31.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612751+0000", 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0107999999999999e-05 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612891+0000", 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013946 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64405 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.386588218, 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025814888000000001, 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64405 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:31.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587379+0000", 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1439999999999999e-06 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587398+0000", 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587405+0000", 2024-09-18T00:58:31.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3740000000000003e-06 2024-09-18T00:58:31.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612931+0000", 2024-09-18T00:58:31.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025526604000000001 2024-09-18T00:58:31.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613008+0000", 2024-09-18T00:58:31.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6508000000000007e-05 2024-09-18T00:58:31.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613190+0000", 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018205200000000001 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64406 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3865362179999998, 2024-09-18T00:58:31.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026015033, 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64406 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:31.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587428+0000", 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0399999999999996e-07 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587431+0000", 2024-09-18T00:58:31.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1870000000000001e-06 2024-09-18T00:58:31.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587441+0000", 2024-09-18T00:58:31.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0349999999999999e-05 2024-09-18T00:58:31.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587447+0000", 2024-09-18T00:58:31.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.083e-06 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613236+0000", 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025788953 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613288+0000", 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2275999999999999e-05 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613442+0000", 2024-09-18T00:58:31.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015358 2024-09-18T00:58:31.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.025 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64407 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3865072190000003, 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026326130999999999, 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64407 2024-09-18T00:58:31.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.027 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:31.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:31.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:31.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:58:31.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587459+0000", 2024-09-18T00:58:31.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3750000000000001e-06 2024-09-18T00:58:31.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587465+0000", 2024-09-18T00:58:31.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7150000000000003e-06 2024-09-18T00:58:31.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587469+0000", 2024-09-18T00:58:31.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8060000000000002e-06 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613488+0000", 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026018645999999999 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613576+0000", 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7892000000000005e-05 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613782+0000", 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000206233 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64408 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3864869759999996, 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026568468000000001, 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64408 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587477+0000", 2024-09-18T00:58:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7500000000000001e-07 2024-09-18T00:58:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587479+0000", 2024-09-18T00:58:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1900000000000002e-06 2024-09-18T00:58:31.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587487+0000", 2024-09-18T00:58:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1869999999999997e-06 2024-09-18T00:58:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587492+0000", 2024-09-18T00:58:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3149999999999999e-06 2024-09-18T00:58:31.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613823+0000", 2024-09-18T00:58:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026330339000000001 2024-09-18T00:58:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613885+0000", 2024-09-18T00:58:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2391999999999995e-05 2024-09-18T00:58:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614045+0000", 2024-09-18T00:58:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015966999999999999 2024-09-18T00:58:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64409 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3864639990000001, 2024-09-18T00:58:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026785499000000001, 2024-09-18T00:58:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64409 2024-09-18T00:58:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587500+0000", 2024-09-18T00:58:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587502+0000", 2024-09-18T00:58:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9420000000000002e-06 2024-09-18T00:58:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587508+0000", 2024-09-18T00:58:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1840000000000004e-06 2024-09-18T00:58:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587513+0000", 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9640000000000002e-06 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614075+0000", 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026561889000000002 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614124+0000", 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9177e-05 2024-09-18T00:58:31.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614285+0000", 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016103100000000001 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64410 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:31.042 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3864440499999997, 2024-09-18T00:58:31.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026987559000000001, 2024-09-18T00:58:31.043 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64410 2024-09-18T00:58:31.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.043 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:31.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:31.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587520+0000", 2024-09-18T00:58:31.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-09-18T00:58:31.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587522+0000", 2024-09-18T00:58:31.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1500000000000002e-06 2024-09-18T00:58:31.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587529+0000", 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5839999999999997e-06 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587534+0000", 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.826e-06 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614317+0000", 2024-09-18T00:58:31.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026783065000000002 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614385+0000", 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7682999999999996e-05 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614507+0000", 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121859 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.048 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64411 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.048 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:31.048 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3864225650000002, 2024-09-18T00:58:31.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027224245000000001, 2024-09-18T00:58:31.048 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64411 2024-09-18T00:58:31.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.049 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:31.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587543+0000", 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:58:31.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587550+0000", 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5810000000000002e-06 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587555+0000", 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0490000000000003e-06 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614543+0000", 2024-09-18T00:58:31.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026988056 2024-09-18T00:58:31.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614619+0000", 2024-09-18T00:58:31.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5932000000000004e-05 2024-09-18T00:58:31.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614765+0000", 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014603899999999999 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64412 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2097152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3864018199999997, 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027478605, 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.053 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64412 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587562+0000", 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6499999999999999e-07 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587564+0000", 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3300000000000001e-06 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587571+0000", 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2949999999999999e-06 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587575+0000", 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6360000000000002e-06 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614798+0000", 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027222860000000002 2024-09-18T00:58:31.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614874+0000", 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5383000000000007e-05 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615040+0000", 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016653600000000001 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64413 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3863810159999996, 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027709421000000001, 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64413 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587583+0000", 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:31.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587585+0000", 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587592+0000", 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9700000000000002e-06 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587597+0000", 2024-09-18T00:58:31.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8289999999999998e-06 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615071+0000", 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027474218000000002 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615143+0000", 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1505000000000001e-05 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615292+0000", 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014900199999999999 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64414 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2457600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3863594780000001, 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027920637000000002, 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64414 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:31.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.08e-07 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587606+0000", 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1299999999999999e-06 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587612+0000", 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1319999999999999e-06 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587618+0000", 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.147e-06 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615323+0000", 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027705423 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615366+0000", 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2914000000000003e-05 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615524+0000", 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015848300000000001 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64415 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.066 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3863351569999995, 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028241265000000002, 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64415 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587631+0000", 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5550000000000001e-06 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587639+0000", 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.807e-06 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587644+0000", 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2630000000000003e-06 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615633+0000", 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027989342 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615703+0000", 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9956000000000006e-05 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615869+0000", 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016597500000000001 2024-09-18T00:58:31.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64416 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2818048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3863118600000002, 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028575605, 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64416 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587652+0000", 2024-09-18T00:58:31.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-18T00:58:31.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587655+0000", 2024-09-18T00:58:31.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-18T00:58:31.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587662+0000", 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7789999999999996e-06 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587668+0000", 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2109999999999997e-06 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615916+0000", 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028248412000000001 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616009+0000", 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2682999999999993e-05 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616227+0000", 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218345 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64417 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3862878490000003, 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028891495, 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64417 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587676+0000", 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.34e-07 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587683+0000", 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8739999999999997e-06 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587692+0000", 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2159999999999995e-06 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587698+0000", 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.716e-06 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616259+0000", 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028561733999999998 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616369+0000", 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110104 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616567+0000", 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000197417 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64418 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3862574050000003, 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029401729000000001, 2024-09-18T00:58:31.080 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64418 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8400000000000004e-07 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587709+0000", 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5390000000000002e-06 2024-09-18T00:58:31.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587717+0000", 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0819999999999999e-06 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587722+0000", 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4140000000000002e-06 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616869+0000", 2024-09-18T00:58:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029146763999999999 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616926+0000", 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7250000000000002e-05 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.617108+0000", 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181096 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:31.085 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:31.085 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:31.085 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:31.085 INFO:teuthology.orchestra.run.smithi042.stdout:2643 trunc from 0x87736b8 to 0x748fbe3 2024-09-18T00:58:31.085 INFO:teuthology.orchestra.run.smithi042.stdout:2644 write 0x6229ceb thru 0x622a51d (0x833 bytes) 2024-09-18T00:58:31.086 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:31.087 INFO:teuthology.orchestra.run.smithi042.stdout:2645 writesame 0x3ba2a16 thru 0x3ba5922 data_size 0x21(0x2f0d bytes) 2024-09-18T00:58:31.087 INFO:teuthology.orchestra.run.smithi042.stdout:2647 read 0x2bd2bfe thru 0x2bd54b1 (0x28b4 bytes) 2024-09-18T00:58:31.087 INFO:teuthology.orchestra.run.smithi042.stdout:2648 trunc from 0x748fbe3 to 0x10dd8b4 2024-09-18T00:58:31.089 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:31.081+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:31.132 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:58:31.133 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:58:31.133 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-18T00:58:31.133 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T00:58:31.133 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T00:58:31.133 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-18T00:58:31.140 INFO:teuthology.orchestra.run.smithi042.stdout:2649 write 0x9244187 thru 0x92532df (0xf159 bytes) 2024-09-18T00:58:31.149 INFO:teuthology.orchestra.run.smithi042.stdout:2650 trunc from 0x92532e0 to 0xba05e03 2024-09-18T00:58:31.158 INFO:teuthology.orchestra.run.smithi042.stdout:2652 read 0xbf2d95 thru 0xbf8dbb (0x6027 bytes) 2024-09-18T00:58:31.159 INFO:teuthology.orchestra.run.smithi042.stdout:2653 punch from 0x7e8a720 to 0x7e93212, (0x8af2 bytes) 2024-09-18T00:58:31.159 INFO:teuthology.orchestra.run.smithi042.stdout:2654 read 0x7b5a416 thru 0x7b68624 (0xe20f bytes) 2024-09-18T00:58:31.159 INFO:teuthology.orchestra.run.smithi042.stdout:2655 read 0x529a7f2 thru 0x52a8e2f (0xe63e bytes) 2024-09-18T00:58:31.159 INFO:teuthology.orchestra.run.smithi042.stdout:2656 write 0x626a321 thru 0x6272ef3 (0x8bd3 bytes) 2024-09-18T00:58:31.162 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:31.162 INFO:teuthology.orchestra.run.smithi042.stdout:2660 punch from 0x7563bfd to 0x7570e1c, (0xd21f bytes) 2024-09-18T00:58:31.175 INFO:teuthology.orchestra.run.smithi042.stdout:2661 writesame 0xec4e4e5 thru 0xec4f851 data_size 0x1(0x136d bytes) 2024-09-18T00:58:31.175 INFO:teuthology.orchestra.run.smithi042.stdout:2662 writesame 0xcbf93b7 thru 0xcc0786b data_size 0x1(0xe4b5 bytes) 2024-09-18T00:58:31.189 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:31.185+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:31.217 INFO:teuthology.orchestra.run.smithi042.stdout:2663 read 0x95d6a5c thru 0x95e17eb (0xad90 bytes) 2024-09-18T00:58:31.218 INFO:teuthology.orchestra.run.smithi042.stdout:2664 read 0x9e2c8be thru 0x9e3b747 (0xee8a bytes) 2024-09-18T00:58:31.230 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:31.230 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:31.230 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:31.230 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:31.238 INFO:teuthology.orchestra.run.smithi042.stdout:2665 trunc from 0xec4f852 to 0x4406759 2024-09-18T00:58:31.245 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:31.282 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "" 2024-09-18T00:58:31.283 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:31.290 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:31.286+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:31.299 INFO:teuthology.orchestra.run.smithi042.stdout:2666 write 0xcf4184e thru 0xcf505da (0xed8d bytes) 2024-09-18T00:58:31.312 INFO:teuthology.orchestra.run.smithi042.stdout:2667 write 0xd0db7e2 thru 0xd0e5404 (0x9c23 bytes) 2024-09-18T00:58:31.330 INFO:teuthology.orchestra.run.smithi042.stdout:2668 trunc from 0xd0e5405 to 0x6e3d93e 2024-09-18T00:58:31.349 INFO:teuthology.orchestra.run.smithi042.stdout:2669 punch from 0x254031e to 0x2547a23, (0x7705 bytes) 2024-09-18T00:58:31.363 INFO:teuthology.orchestra.run.smithi042.stdout:2670 write 0x8c29f83 thru 0x8c313ea (0x7468 bytes) 2024-09-18T00:58:31.370 INFO:teuthology.orchestra.run.smithi042.stdout:2671 read 0xb615b2 thru 0xb69237 (0x7c86 bytes) 2024-09-18T00:58:31.391 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:31.386+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:31.395 INFO:teuthology.orchestra.run.smithi042.stdout:2673 write 0xebb5584 thru 0xebc012d (0xabaa bytes) 2024-09-18T00:58:31.401 INFO:teuthology.orchestra.run.smithi042.stdout:2674 read 0xde5b4e3 thru 0xde6873f (0xd25d bytes) 2024-09-18T00:58:31.402 INFO:teuthology.orchestra.run.smithi042.stdout:2675 read 0xd1e2ab1 thru 0xd1e4f4f (0x249f bytes) 2024-09-18T00:58:31.403 INFO:teuthology.orchestra.run.smithi042.stdout:2676 read 0xc22fa1d thru 0xc23d671 (0xdc55 bytes) 2024-09-18T00:58:31.403 INFO:teuthology.orchestra.run.smithi042.stdout:2677 punch from 0x75f0ba7 to 0x75f4e49, (0x42a2 bytes) 2024-09-18T00:58:31.403 INFO:teuthology.orchestra.run.smithi042.stdout:2678 writesame 0x8ed29aa thru 0x8eddef8 data_size 0x1(0xb54f bytes) 2024-09-18T00:58:31.425 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:31.425 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:31.425 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:31.426 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:31.426 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:31.428 INFO:teuthology.orchestra.run.smithi042.stdout:2679 read 0xb7b1053 thru 0xb7b572f (0x46dd bytes) 2024-09-18T00:58:31.428 INFO:teuthology.orchestra.run.smithi042.stdout:2680 write 0x990666b thru 0x990878a (0x2120 bytes) 2024-09-18T00:58:31.434 INFO:teuthology.orchestra.run.smithi042.stdout:2682 punch from 0x70eb78f to 0x70f5033, (0x98a4 bytes) 2024-09-18T00:58:31.441 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:31.467 INFO:teuthology.orchestra.run.smithi042.stdout:2686 trunc from 0xebc012e to 0x29a4ada 2024-09-18T00:58:31.493 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:31.486+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout:2687 write 0xa18e154 thru 0xa19705e (0x8f0b bytes) 2024-09-18T00:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout:2688 write 0x1587561 thru 0x1591097 (0x9b37 bytes) 2024-09-18T00:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout:2689 writesame 0x24cb437 thru 0x24ccec2 data_size 0x2(0x1a8c bytes) 2024-09-18T00:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout:2690 write 0xa08e0d3 thru 0xa091954 (0x3882 bytes) 2024-09-18T00:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout:2691 punch from 0x76f8e2 to 0x772864, (0x2f82 bytes) 2024-09-18T00:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout:2692 read 0x759e642 thru 0x75acc11 (0xe5d0 bytes) 2024-09-18T00:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout:2694 punch from 0x45ab45d to 0x45b58a3, (0xa446 bytes) 2024-09-18T00:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout:2695 trunc from 0xa19705f to 0xc9b6802 2024-09-18T00:58:31.565 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:31.565 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:31.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:31.565 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:31.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.565 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1670 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1335296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.565 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:31.565 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.394388028999998, 2024-09-18T00:58:31.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083399740999999999, 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1670 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:31.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:31.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166573+0000", 2024-09-18T00:58:31.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1829999999999999e-06 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166578+0000", 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0030000000000002e-06 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166594+0000", 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6470999999999999e-05 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166607+0000", 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2831e-05 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249706+0000", 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083099022999999994 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249843+0000", 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136518 2024-09-18T00:58:31.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249971+0000", 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012871200000000001 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1671 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1376256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.394337252, 2024-09-18T00:58:31.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083700574, 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1671 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:31.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166623+0000", 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.066e-06 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166641+0000", 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7728e-05 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166659+0000", 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7850999999999999e-05 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166672+0000", 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3020000000000001e-05 2024-09-18T00:58:31.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250034+0000", 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083362085000000002 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250145+0000", 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111045 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250323+0000", 2024-09-18T00:58:31.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017777900000000001 2024-09-18T00:58:31.577 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.577 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.577 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.577 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1672 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.577 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:31.577 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.394263793, 2024-09-18T00:58:31.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083853480999999994, 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1672 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:31.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166697+0000", 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0920000000000001e-06 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166702+0000", 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0799999999999996e-06 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166718+0000", 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6167999999999998e-05 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166745+0000", 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7177000000000001e-05 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250382+0000", 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083636792000000001 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250476+0000", 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250549+0000", 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3282999999999996e-05 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1673 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.394196684999997, 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084151880999999998, 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.582 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1673 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166764+0000", 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166770+0000", 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5969999999999999e-06 2024-09-18T00:58:31.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166786+0000", 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6439999999999998e-05 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166799+0000", 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2934e-05 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250605+0000", 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083806357999999997 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250724+0000", 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011821500000000001 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250915+0000", 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000191251 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1674 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.394097100000003, 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084239101999999996, 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.587 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1674 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166864+0000", 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3629999999999999e-06 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166883+0000", 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9083e-05 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166902+0000", 2024-09-18T00:58:31.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8711000000000001e-05 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166915+0000", 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3375e-05 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250962+0000", 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084047254000000002 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251029+0000", 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6315000000000005e-05 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251102+0000", 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3001000000000001e-05 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1675 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1540096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:31.591 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.394025249000002, 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084536891000000003, 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1675 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166936+0000", 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2720000000000001e-06 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166941+0000", 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7370000000000003e-06 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166958+0000", 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166972+0000", 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3569e-05 2024-09-18T00:58:31.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251159+0000", 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084187467000000002 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251300+0000", 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014059200000000001 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251471+0000", 2024-09-18T00:58:31.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017168000000000001 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1676 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.393972368999997, 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084738637000000006, 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.596 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1676 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166988+0000", 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166993+0000", 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8339999999999998e-06 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167009+0000", 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6028e-05 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167022+0000", 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2675e-05 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251517+0000", 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084494975999999999 2024-09-18T00:58:31.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251576+0000", 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8644999999999998e-05 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251726+0000", 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015039 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1677 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.393920698000002, 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085055403000000002, 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1677 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.601 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167040+0000", 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167046+0000", 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167062+0000", 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6354e-05 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167075+0000", 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2965e-05 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251789+0000", 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084713940000000001 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251916+0000", 2024-09-18T00:58:31.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000127176 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252094+0000", 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017833599999999999 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1678 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:31.605 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.393868935, 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085316746999999998, 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1678 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167092+0000", 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167097+0000", 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7029999999999997e-06 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167113+0000", 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167125+0000", 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2517000000000001e-05 2024-09-18T00:58:31.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252157+0000", 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085031933000000004 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252277+0000", 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119708 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252407+0000", 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013031800000000001 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1679 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.393818216, 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085557195000000003, 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.610 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1679 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:31.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167143+0000", 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167170+0000", 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7523000000000001e-05 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167189+0000", 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8530999999999999e-05 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167215+0000", 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6676999999999999e-05 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252445+0000", 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085229416000000002 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252529+0000", 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4740000000000005e-05 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252699+0000", 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016920800000000001 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.614 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1680 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.393724759000001, 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085810110999999994, 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1680 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167236+0000", 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.048e-06 2024-09-18T00:58:31.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167242+0000", 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6459999999999998e-06 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167259+0000", 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.696e-05 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167271+0000", 2024-09-18T00:58:31.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.292e-05 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252839+0000", 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085567292000000003 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252924+0000", 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5511000000000002e-05 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253045+0000", 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120734 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1681 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.393618662999998, 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085887343000000005, 2024-09-18T00:58:31.619 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1681 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167342+0000", 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167348+0000", 2024-09-18T00:58:31.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4519999999999996e-06 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167372+0000", 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4261999999999999e-05 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167400+0000", 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253091+0000", 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085690895000000003 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253154+0000", 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3262000000000005e-05 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253228+0000", 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4318000000000002e-05 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.623 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1682 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.393540582999997, 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.086061112999999995, 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.624 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1682 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168114+0000", 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00069532600000000002 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168123+0000", 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8319999999999995e-06 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168147+0000", 2024-09-18T00:58:31.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3536999999999998e-05 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168163+0000", 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6317e-05 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253274+0000", 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085111390999999995 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253362+0000", 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7384000000000002e-05 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253480+0000", 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011832600000000001 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.628 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1683 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1867776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.392763451999997, 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085456638000000001, 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1683 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169938+0000", 2024-09-18T00:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017419079999999999 2024-09-18T00:58:31.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169951+0000", 2024-09-18T00:58:31.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2534999999999999e-05 2024-09-18T00:58:31.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169990+0000", 2024-09-18T00:58:31.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9004999999999997e-05 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170011+0000", 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.156e-05 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253534+0000", 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083522759000000002 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253585+0000", 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1316999999999997e-05 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253653+0000", 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7553999999999997e-05 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1684 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.633 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.390913816000001, 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083787188999999998, 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1684 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:31.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170048+0000", 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.674e-06 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170055+0000", 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3549999999999999e-06 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170073+0000", 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7662000000000001e-05 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170087+0000", 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4229e-05 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253701+0000", 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083614346000000006 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253752+0000", 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0760000000000002e-05 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253833+0000", 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1162999999999995e-05 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.637 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1685 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.390853018999998, 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083900785000000005, 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.638 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1685 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170108+0000", 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.229e-06 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170114+0000", 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0079999999999999e-06 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170132+0000", 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7734e-05 2024-09-18T00:58:31.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170145+0000", 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2977e-05 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253866+0000", 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083721208000000005 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253934+0000", 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.834e-05 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254007+0000", 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3288999999999996e-05 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.642 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1686 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.390797808000002, 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084083526000000006, 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1686 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170163+0000", 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1909999999999999e-06 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170180+0000", 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7323e-05 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170198+0000", 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7723e-05 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170212+0000", 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3645999999999999e-05 2024-09-18T00:58:31.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254050+0000", 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083838115000000005 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254136+0000", 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6106999999999994e-05 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254245+0000", 2024-09-18T00:58:31.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010942099999999999 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1687 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.390729506, 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084186574, 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.647 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1687 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170647+0000", 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416649 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170655+0000", 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6510000000000007e-06 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170683+0000", 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7093999999999998e-05 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170701+0000", 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8383e-05 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254283+0000", 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083582091999999997 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254339+0000", 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6187999999999998e-05 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254417+0000", 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7516999999999997e-05 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1688 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.390233791999997, 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083931026000000006, 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1688 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.652 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173553+0000", 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002827333 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173566+0000", 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3276e-05 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173603+0000", 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6177000000000003e-05 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173628+0000", 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5343000000000001e-05 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254460+0000", 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.080831745999999996 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254547+0000", 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7045000000000004e-05 2024-09-18T00:58:31.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254657+0000", 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011010600000000001 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.78609659, 2024-09-18T00:58:31.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:58:31.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:58:31.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:58:31.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:31.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:58:31.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:58:31.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.662 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:31.662 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:31.662 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:31.664 INFO:teuthology.orchestra.run.smithi042.stdout:2696 read 0x21fe700 thru 0x2209273 (0xab74 bytes) 2024-09-18T00:58:31.664 INFO:teuthology.orchestra.run.smithi042.stdout:2697 write 0x41c38ff thru 0x41c45df (0xce1 bytes) 2024-09-18T00:58:31.664 INFO:teuthology.orchestra.run.smithi042.stdout:2699 read 0x975f0b0 thru 0x9762ec9 (0x3e1a bytes) 2024-09-18T00:58:31.664 INFO:teuthology.orchestra.run.smithi042.stdout:2700 writesame 0x6bde51d thru 0x6be85b0 data_size 0x56(0xa094 bytes) 2024-09-18T00:58:31.664 INFO:teuthology.orchestra.run.smithi042.stdout:2701 trunc from 0xc9b6802 to 0x4550fac 2024-09-18T00:58:31.664 INFO:teuthology.orchestra.run.smithi042.stdout:2702 trunc from 0x4550fac to 0xe98e3a0 2024-09-18T00:58:31.664 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:31.665 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:31.657+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:31.672 INFO:teuthology.orchestra.run.smithi042.stdout:2705 write 0x5ba1a5b thru 0x5ba659b (0x4b41 bytes) 2024-09-18T00:58:31.676 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:31.676 INFO:teuthology.orchestra.run.smithi042.stdout:2708 write 0x10a2dd6 thru 0x10ae399 (0xb5c4 bytes) 2024-09-18T00:58:31.680 INFO:teuthology.orchestra.run.smithi042.stdout:2709 read 0x5765c3d thru 0x5771128 (0xb4ec bytes) 2024-09-18T00:58:31.681 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:31.682 INFO:teuthology.orchestra.run.smithi042.stdout:2711 write 0x25189ec thru 0x2521245 (0x885a bytes) 2024-09-18T00:58:31.685 INFO:teuthology.orchestra.run.smithi042.stdout:2712 read 0x33b693b thru 0x33be85d (0x7f23 bytes) 2024-09-18T00:58:31.687 INFO:teuthology.orchestra.run.smithi042.stdout:2713 write 0x1ad8a35 thru 0x1ae1396 (0x8962 bytes) 2024-09-18T00:58:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:2714 punch from 0x4b43f62 to 0x4b4c486, (0x8524 bytes) 2024-09-18T00:58:31.691 INFO:teuthology.orchestra.run.smithi042.stdout:2715 writesame 0x1b0c3 thru 0x299b9 data_size 0x1(0xe8f7 bytes) 2024-09-18T00:58:31.739 INFO:teuthology.orchestra.run.smithi042.stdout:2718 punch from 0x2dc7a05 to 0x2dd7295, (0xf890 bytes) 2024-09-18T00:58:31.739 INFO:teuthology.orchestra.run.smithi042.stdout:2719 write 0x4921ab9 thru 0x492bba4 (0xa0ec bytes) 2024-09-18T00:58:31.743 INFO:teuthology.orchestra.run.smithi042.stdout:2720 trunc from 0xe98e3a0 to 0xea12c1a 2024-09-18T00:58:31.756 INFO:teuthology.orchestra.run.smithi042.stdout:2723 writesame 0x111401d thru 0x1119003 data_size 0x1(0x4fe7 bytes) 2024-09-18T00:58:31.764 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:31.757+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:31.784 INFO:teuthology.orchestra.run.smithi042.stdout:2724 write 0x46a7955 thru 0x46a8a8e (0x113a bytes) 2024-09-18T00:58:31.801 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:31.807 INFO:teuthology.orchestra.run.smithi042.stdout:2725 trunc from 0xea12c1a to 0x7670dfc 2024-09-18T00:58:31.819 INFO:teuthology.orchestra.run.smithi042.stdout:2726 write 0x1cf97d5 thru 0x1cffbfd (0x6429 bytes) 2024-09-18T00:58:31.822 INFO:teuthology.orchestra.run.smithi042.stdout:2727 writesame 0x5ce21ca thru 0x5cec658 data_size 0x1(0xa48f bytes) 2024-09-18T00:58:31.836 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:31.836 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:31.836 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:31.836 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:31.851 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:31.864 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:31.858+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:31.865 INFO:teuthology.orchestra.run.smithi042.stdout:2728 write 0x588be8e thru 0x5899298 (0xd40b bytes) 2024-09-18T00:58:31.868 INFO:teuthology.orchestra.run.smithi042.stdout:2730 read 0x431f81f thru 0x432d703 (0xdee5 bytes) 2024-09-18T00:58:31.869 INFO:teuthology.orchestra.run.smithi042.stdout:2732 punch from 0x75cace9 to 0x75d8a28, (0xdd3f bytes) 2024-09-18T00:58:31.870 INFO:teuthology.orchestra.run.smithi042.stdout:2734 write 0x1b2edca thru 0x1b39df9 (0xb030 bytes) 2024-09-18T00:58:31.884 INFO:teuthology.orchestra.run.smithi042.stdout:2735 trunc from 0x7670dfc to 0xddf9f3e 2024-09-18T00:58:31.904 INFO:teuthology.orchestra.run.smithi042.stdout:2738 trunc from 0xddf9f3e to 0x8a20065 2024-09-18T00:58:31.915 INFO:teuthology.orchestra.run.smithi042.stdout:2739 punch from 0x638083c to 0x638f816, (0xefda bytes) 2024-09-18T00:58:31.915 INFO:teuthology.orchestra.run.smithi042.stdout:2740 writesame 0x4bc815a thru 0x4bcf2c3 data_size 0x9(0x716a bytes) 2024-09-18T00:58:31.933 INFO:teuthology.orchestra.run.smithi042.stdout:2742 write 0x97df8cc thru 0x97dfd2b (0x460 bytes) 2024-09-18T00:58:31.938 INFO:teuthology.orchestra.run.smithi042.stdout:2744 writesame 0x3a2a90b thru 0x3a2cab8 data_size 0x9(0x21ae bytes) 2024-09-18T00:58:31.941 INFO:teuthology.orchestra.run.smithi042.stdout:2746 read 0x5246e1c thru 0x5252233 (0xb418 bytes) 2024-09-18T00:58:31.951 INFO:teuthology.orchestra.run.smithi042.stdout:2747 writesame 0xabf53f3 thru 0xabffe03 data_size 0x11(0xaa11 bytes) 2024-09-18T00:58:31.953 INFO:teuthology.orchestra.run.smithi042.stdout:2748 write 0x2808a8d thru 0x2818a80 (0xfff4 bytes) 2024-09-18T00:58:31.956 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:31.956 INFO:teuthology.orchestra.run.smithi042.stdout:2750 punch from 0xab2c310 to 0xab370bc, (0xadac bytes) 2024-09-18T00:58:31.956 INFO:teuthology.orchestra.run.smithi042.stdout:2751 read 0x9eacc1a thru 0x9eb8deb (0xc1d2 bytes) 2024-09-18T00:58:31.956 INFO:teuthology.orchestra.run.smithi042.stdout:2753 read 0x3fd400a thru 0x3fd9ebd (0x5eb4 bytes) 2024-09-18T00:58:31.965 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:31.961+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:31.969 INFO:teuthology.orchestra.run.smithi042.stdout:2754 trunc from 0xabffe04 to 0x22d3899 2024-09-18T00:58:31.978 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:31.978 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:31.991 INFO:teuthology.orchestra.run.smithi042.stdout:2755 write 0x26ba5aa thru 0x26bf569 (0x4fc0 bytes) 2024-09-18T00:58:31.993 INFO:teuthology.orchestra.run.smithi042.stdout:2756 read 0x9113e4 thru 0x919f3e (0x8b5b bytes) 2024-09-18T00:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:31.999 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout:2757 write 0x37b8a8e thru 0x37be9d3 (0x5f46 bytes) 2024-09-18T00:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout:2760 clone 24 order 24 su 262144 sc 2 2024-09-18T00:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:32.017 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:32.023 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:32.024 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:32.024 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:32.024 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:32.024 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:32.034 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:32.038 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:32.052 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:32.071 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:32.085 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:32.081+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:32.088 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:32.088 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:32.106 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:32.106 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:32.161 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:32.161 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:32.167 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:32.161+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:32.174 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:32.268 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:32.261+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:32.283 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:58:32.348 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:32.348 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:32.362 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:32.409 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:32.405+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:32.425 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone23 from 0x37be9d4 (overlap 0x10dd8b4) to 0xf2dca2 2024-09-18T00:58:32.469 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:32.465+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:32.534 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-18T00:58:32.543 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:32.543 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:32.548 INFO:teuthology.orchestra.run.smithi042.stdout:2761 punch from 0x19d1cc to 0x1a73fc, (0xa230 bytes) 2024-09-18T00:58:32.559 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout:2762 write 0x4d446c6 thru 0x4d4ff7d (0xb8b8 bytes) 2024-09-18T00:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout:2763 write 0x3720cbb thru 0x372d0e7 (0xc42d bytes) 2024-09-18T00:58:32.570 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:32.565+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout:2764 write 0xcfbca90 thru 0xcfcb3dd (0xe94e bytes) 2024-09-18T00:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout:2765 writesame 0x4a4264d thru 0x4a50269 data_size 0x9(0xdc1d bytes) 2024-09-18T00:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout:2766 read 0x8697532 thru 0x869a8ff (0x33ce bytes) 2024-09-18T00:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout:2768 read 0x41f19bb thru 0x41fa10f (0x8755 bytes) 2024-09-18T00:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout:2769 trunc from 0xcfcb3de to 0x23b527b 2024-09-18T00:58:32.668 INFO:teuthology.orchestra.run.smithi042.stdout:2770 punch from 0x2125c41 to 0x2133292, (0xd651 bytes) 2024-09-18T00:58:32.668 INFO:teuthology.orchestra.run.smithi042.stdout:2771 punch from 0x1536f5 to 0x15ab39, (0x7444 bytes) 2024-09-18T00:58:32.671 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:32.665+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:32.685 INFO:teuthology.orchestra.run.smithi042.stdout:2773 trunc from 0x23b527b to 0x9bddc30 2024-09-18T00:58:32.696 INFO:teuthology.orchestra.run.smithi042.stdout:2774 read 0x4dabd32 thru 0x4db35fa (0x78c9 bytes) 2024-09-18T00:58:32.719 INFO:teuthology.orchestra.run.smithi042.stdout:2775 writesame 0xecfff03 thru 0xed0f35a data_size 0x1c(0xf458 bytes) 2024-09-18T00:58:32.719 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:32.719 INFO:teuthology.orchestra.run.smithi042.stdout:2778 writesame 0x14aa575 thru 0x14aa7e1 data_size 0x9(0x26d bytes) 2024-09-18T00:58:32.727 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:32.727 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:32.742 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:32.780 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:32.780 INFO:teuthology.orchestra.run.smithi042.stdout:2781 read 0x6e39c75 thru 0x6e49756 (0xfae2 bytes) 2024-09-18T00:58:32.780 INFO:teuthology.orchestra.run.smithi042.stdout:2782 write 0x403f8bc thru 0x404e63e (0xed83 bytes) 2024-09-18T00:58:32.788 INFO:teuthology.orchestra.run.smithi042.stdout:2783 punch from 0x25d20c7 to 0x25de9a2, (0xc8db bytes) 2024-09-18T00:58:32.788 INFO:teuthology.orchestra.run.smithi042.stdout:2786 writesame 0xdd56b3a thru 0xdd600c7 data_size 0x1b(0x958e bytes) 2024-09-18T00:58:32.793 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:32.789+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:32.795 INFO:teuthology.orchestra.run.smithi042.stdout:2788 clone 25 order 19 su 65536 sc 9 2024-09-18T00:58:32.865 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:32.865 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:32.874 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:32.869+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:32.879 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:32.975 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:32.969+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:33.041 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:33.041 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:33.055 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:33.076 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:33.069+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:33.176 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:33.173+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:33.223 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:33.223 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:33.237 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:33.270 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:58:33.277 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:33.273+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:33.377 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:33.373+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:33.387 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:33.387 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:33.405 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:33.415 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone24 from 0xed0f35b (overlap 0x23b527b) to 0x3921d9 2024-09-18T00:58:33.478 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:33.473+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:33.545 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:33.545 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:33.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:33.602 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:33.597+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:33.680 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:33.673+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:33.715 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-18T00:58:33.745 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:33.745 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:33.762 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:33.784 INFO:teuthology.orchestra.run.smithi042.stdout:2789 write 0xa7a9b17 thru 0xa7b7916 (0xde00 bytes) 2024-09-18T00:58:33.785 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:33.777+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:33.801 INFO:teuthology.orchestra.run.smithi042.stdout:2790 writesame 0x2833ae4 thru 0x283845b data_size 0x4(0x4978 bytes) 2024-09-18T00:58:33.811 INFO:teuthology.orchestra.run.smithi042.stdout:2791 write 0xf84e43 thru 0xf87c72 (0x2e30 bytes) 2024-09-18T00:58:33.821 INFO:teuthology.orchestra.run.smithi042.stdout:2792 writesame 0xa6ef1b9 thru 0xa6f7f02 data_size 0x5(0x8d4a bytes) 2024-09-18T00:58:33.836 INFO:teuthology.orchestra.run.smithi042.stdout:2793 write 0x5fc8b01 thru 0x5fcce89 (0x4389 bytes) 2024-09-18T00:58:33.845 INFO:teuthology.orchestra.run.smithi042.stdout:2794 write 0x587e298 thru 0x5883408 (0x5171 bytes) 2024-09-18T00:58:33.856 INFO:teuthology.orchestra.run.smithi042.stdout:2795 read 0x3f257eb thru 0x3f2b521 (0x5d37 bytes) 2024-09-18T00:58:33.856 INFO:teuthology.orchestra.run.smithi042.stdout:2796 punch from 0x63ef51f to 0x63f3d4e, (0x482f bytes) 2024-09-18T00:58:33.857 INFO:teuthology.orchestra.run.smithi042.stdout:2798 punch from 0xa874be5 to 0xa87a675, (0x5a90 bytes) 2024-09-18T00:58:33.857 INFO:teuthology.orchestra.run.smithi042.stdout:2799 writesame 0x995a538 thru 0x9966ac3 data_size 0x5e(0xc58c bytes) 2024-09-18T00:58:33.868 INFO:teuthology.orchestra.run.smithi042.stdout:2800 punch from 0x12687e8 to 0x1276bb5, (0xe3cd bytes) 2024-09-18T00:58:33.882 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:33.877+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:33.890 INFO:teuthology.orchestra.run.smithi042.stdout:2801 trunc from 0xed0f35b to 0x19a47e1 2024-09-18T00:58:33.930 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:33.930 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:33.952 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:33.969 INFO:teuthology.orchestra.run.smithi042.stdout:2802 write 0xd495bbc thru 0xd495cef (0x134 bytes) 2024-09-18T00:58:33.976 INFO:teuthology.orchestra.run.smithi042.stdout:2804 read 0xccc3fcf thru 0xccd1ee6 (0xdf18 bytes) 2024-09-18T00:58:33.984 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:33.977+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:33.992 INFO:teuthology.orchestra.run.smithi042.stdout:2806 trunc from 0xd495cf0 to 0x33fad87 2024-09-18T00:58:34.010 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.033 INFO:teuthology.orchestra.run.smithi042.stdout:2808 writesame 0xc29ad7c thru 0xc2a940e data_size 0x1(0xe693 bytes) 2024-09-18T00:58:34.034 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.052 INFO:teuthology.orchestra.run.smithi042.stdout:2810 trunc from 0xc2a940f to 0x4f007f 2024-09-18T00:58:34.084 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:34.077+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:34.100 INFO:teuthology.orchestra.run.smithi042.stdout:2811 punch from 0x1832a6 to 0x18d041, (0x9d9b bytes) 2024-09-18T00:58:34.109 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:34.109 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:34.124 INFO:teuthology.orchestra.run.smithi042.stdout:2812 writesame 0x9f522c1 thru 0x9f5981d data_size 0x5(0x755d bytes) 2024-09-18T00:58:34.124 INFO:teuthology.orchestra.run.smithi042.stdout:2814 read 0x751b7ff thru 0x7521994 (0x6196 bytes) 2024-09-18T00:58:34.124 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:34.127 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:34.136 INFO:teuthology.orchestra.run.smithi042.stdout:2815 write 0xb5a8d76 thru 0xb5aa870 (0x1afb bytes) 2024-09-18T00:58:34.143 INFO:teuthology.orchestra.run.smithi042.stdout:2816 read 0xb4da562 thru 0xb4dcbb3 (0x2652 bytes) 2024-09-18T00:58:34.143 INFO:teuthology.orchestra.run.smithi042.stdout:2818 write 0x4def87a thru 0x4dfa5dd (0xad64 bytes) 2024-09-18T00:58:34.154 INFO:teuthology.orchestra.run.smithi042.stdout:2819 punch from 0xb562fbe to 0xb568214, (0x5256 bytes) 2024-09-18T00:58:34.171 INFO:teuthology.orchestra.run.smithi042.stdout:2821 trunc from 0xb5aa871 to 0x1d069b9 2024-09-18T00:58:34.185 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:34.181+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:34.191 INFO:teuthology.orchestra.run.smithi042.stdout:2823 punch from 0x11dd8fd to 0x11e8bdc, (0xb2df bytes) 2024-09-18T00:58:34.192 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.192 INFO:teuthology.orchestra.run.smithi042.stdout:2825 punch from 0x12e310c to 0x12eb926, (0x881a bytes) 2024-09-18T00:58:34.192 INFO:teuthology.orchestra.run.smithi042.stdout:2826 read 0x5d8191 thru 0x5df9e8 (0x7858 bytes) 2024-09-18T00:58:34.209 INFO:teuthology.orchestra.run.smithi042.stdout:2827 writesame 0x6c44c2e thru 0x6c54b44 data_size 0x13(0xff17 bytes) 2024-09-18T00:58:34.209 INFO:teuthology.orchestra.run.smithi042.stdout:2829 read 0x34b7d thru 0x387f8 (0x3c7c bytes) 2024-09-18T00:58:34.210 INFO:teuthology.orchestra.run.smithi042.stdout:2830 writesame 0x694710a thru 0x694ec61 data_size 0x4(0x7b58 bytes) 2024-09-18T00:58:34.227 INFO:teuthology.orchestra.run.smithi042.stdout:2833 read 0x2e4e13 thru 0x2f2337 (0xd525 bytes) 2024-09-18T00:58:34.230 INFO:teuthology.orchestra.run.smithi042.stdout:2834 write 0x67a6388 thru 0x67ac735 (0x63ae bytes) 2024-09-18T00:58:34.240 INFO:teuthology.orchestra.run.smithi042.stdout:2835 read 0x2197582 thru 0x219d6f9 (0x6178 bytes) 2024-09-18T00:58:34.249 INFO:teuthology.orchestra.run.smithi042.stdout:2836 trunc from 0x6c54b45 to 0xc46524c 2024-09-18T00:58:34.258 INFO:teuthology.orchestra.run.smithi042.stdout:2838 write 0x3b39647 thru 0x3b414b4 (0x7e6e bytes) 2024-09-18T00:58:34.266 INFO:teuthology.orchestra.run.smithi042.stdout:2839 write 0x5244171 thru 0x524c0de (0x7f6e bytes) 2024-09-18T00:58:34.272 INFO:teuthology.orchestra.run.smithi042.stdout:2842 write 0xae889a4 thru 0xae9323c (0xa899 bytes) 2024-09-18T00:58:34.279 INFO:teuthology.orchestra.run.smithi042.stdout:2843 punch from 0x84502f2 to 0x8450796, (0x4a4 bytes) 2024-09-18T00:58:34.279 INFO:teuthology.orchestra.run.smithi042.stdout:2845 write 0x5d34c5d thru 0x5d44748 (0xfaec bytes) 2024-09-18T00:58:34.283 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:34.283 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:34.286 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:34.281+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:34.287 INFO:teuthology.orchestra.run.smithi042.stdout:2847 punch from 0x23ae2d0 to 0x23b9214, (0xaf44 bytes) 2024-09-18T00:58:34.287 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.289 INFO:teuthology.orchestra.run.smithi042.stdout:2849 trunc from 0xc46524c to 0xdc9c7f3 2024-09-18T00:58:34.290 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:34.290 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:34.297 INFO:teuthology.orchestra.run.smithi042.stdout:2851 write 0xfd204d thru 0xfdd395 (0xb349 bytes) 2024-09-18T00:58:34.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:34.306 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.306 INFO:teuthology.orchestra.run.smithi042.stdout:2853 writesame 0x5a8516b thru 0x5a8d458 data_size 0x1(0x82ee bytes) 2024-09-18T00:58:34.308 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:34.309 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:34.328 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:34.328 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:34.328 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.328 INFO:teuthology.orchestra.run.smithi042.stdout:2856 read 0x4eab725 thru 0x4ead3ab (0x1c87 bytes) 2024-09-18T00:58:34.329 INFO:teuthology.orchestra.run.smithi042.stdout:2857 write 0x24d2a32 thru 0x24d9037 (0x6606 bytes) 2024-09-18T00:58:34.334 INFO:teuthology.orchestra.run.smithi042.stdout:2860 read 0x456c9ad thru 0x4572e53 (0x64a7 bytes) 2024-09-18T00:58:34.334 INFO:teuthology.orchestra.run.smithi042.stdout:2861 writesame 0x4f16dd thru 0x4fc03f data_size 0x1(0xa963 bytes) 2024-09-18T00:58:34.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:34.346 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:34.358 INFO:teuthology.orchestra.run.smithi042.stdout:2862 write 0x7baa94 thru 0x7bec3a (0x41a7 bytes) 2024-09-18T00:58:34.373 INFO:teuthology.orchestra.run.smithi042.stdout:2863 trunc from 0xdc9c7f3 to 0x6546311 2024-09-18T00:58:34.383 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:34.383 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:34.386 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:34.381+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:34.390 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.404 INFO:teuthology.orchestra.run.smithi042.stdout:2866 write 0xac42e99 thru 0xac4c040 (0x91a8 bytes) 2024-09-18T00:58:34.410 INFO:teuthology.orchestra.run.smithi042.stdout:2867 read 0x261cf3f thru 0x2623174 (0x6236 bytes) 2024-09-18T00:58:34.410 INFO:teuthology.orchestra.run.smithi042.stdout:2869 writesame 0x35fbc4d thru 0x360183e data_size 0x3(0x5bf2 bytes) 2024-09-18T00:58:34.420 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:34.421 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:34.421 INFO:teuthology.orchestra.run.smithi042.stdout:2870 write 0x507a421 thru 0x507f8b2 (0x5492 bytes) 2024-09-18T00:58:34.427 INFO:teuthology.orchestra.run.smithi042.stdout:2871 punch from 0x8757f05 to 0x876371c, (0xb817 bytes) 2024-09-18T00:58:34.427 INFO:teuthology.orchestra.run.smithi042.stdout:2873 write 0x4afbd77 thru 0x4b0bcc5 (0xff4f bytes) 2024-09-18T00:58:34.450 INFO:teuthology.orchestra.run.smithi042.stdout:2875 write 0xed826ca thru 0xed905fd (0xdf34 bytes) 2024-09-18T00:58:34.451 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:34.452 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:34.459 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:34.459 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:34.459 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:34.459 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:34.459 INFO:teuthology.orchestra.run.smithi042.stdout:2877 read 0x92d12a3 thru 0x92de571 (0xd2cf bytes) 2024-09-18T00:58:34.459 INFO:teuthology.orchestra.run.smithi042.stdout:2879 writesame 0x14d8b29 thru 0x14e7e93 data_size 0x67(0xf36b bytes) 2024-09-18T00:58:34.467 INFO:teuthology.orchestra.run.smithi042.stdout:2880 read 0x364d6d1 thru 0x364fbe6 (0x2516 bytes) 2024-09-18T00:58:34.473 INFO:teuthology.orchestra.run.smithi042.stdout:2881 trunc from 0xed905fe to 0xbaaaeba 2024-09-18T00:58:34.475 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:34.476 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:34.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:34.488 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:34.481+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:34.493 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.493 INFO:teuthology.orchestra.run.smithi042.stdout:2883 read 0x327f194 thru 0x32861b2 (0x701f bytes) 2024-09-18T00:58:34.495 INFO:teuthology.orchestra.run.smithi042.stdout:2886 trunc from 0xbaaaeba to 0xc94959d 2024-09-18T00:58:34.503 INFO:teuthology.orchestra.run.smithi042.stdout:2887 write 0xa241b4a thru 0xa246db8 (0x526f bytes) 2024-09-18T00:58:34.509 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.509 INFO:teuthology.orchestra.run.smithi042.stdout:2889 write 0x18dcfc6 thru 0x18dd54e (0x589 bytes) 2024-09-18T00:58:34.514 INFO:teuthology.orchestra.run.smithi042.stdout:2890 punch from 0x3719c73 to 0x372014e, (0x64db bytes) 2024-09-18T00:58:34.514 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.514 INFO:teuthology.orchestra.run.smithi042.stdout:2893 write 0xdcec77 thru 0xdd89ed (0x9d77 bytes) 2024-09-18T00:58:34.521 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.521 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.521 INFO:teuthology.orchestra.run.smithi042.stdout:2897 writesame 0x7ba30b0 thru 0x7bab1a4 data_size 0x1(0x80f5 bytes) 2024-09-18T00:58:34.567 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.567 INFO:teuthology.orchestra.run.smithi042.stdout:2899 write 0x5d61c56 thru 0x5d6f528 (0xd8d3 bytes) 2024-09-18T00:58:34.575 INFO:teuthology.orchestra.run.smithi042.stdout:2900 punch from 0x6f11b73 to 0x6f15854, (0x3ce1 bytes) 2024-09-18T00:58:34.575 INFO:teuthology.orchestra.run.smithi042.stdout:2901 punch from 0x2de895a to 0x2ded225, (0x48cb bytes) 2024-09-18T00:58:34.575 INFO:teuthology.orchestra.run.smithi042.stdout:2902 write 0x1480520 thru 0x1490200 (0xfce1 bytes) 2024-09-18T00:58:34.588 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:34.581+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:34.611 INFO:teuthology.orchestra.run.smithi042.stdout:2903 trunc from 0xc94959d to 0x4df50b 2024-09-18T00:58:34.613 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:34.614 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:34.614 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:34.614 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:34.614 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:34.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:34.657 INFO:teuthology.orchestra.run.smithi042.stdout:2904 read 0x1e470b thru 0x1efcd7 (0xb5cd bytes) 2024-09-18T00:58:34.659 INFO:teuthology.orchestra.run.smithi042.stdout:2906 read 0x41b8fb thru 0x41fb9d (0x42a3 bytes) 2024-09-18T00:58:34.660 INFO:teuthology.orchestra.run.smithi042.stdout:2907 read 0x35ae00 thru 0x368848 (0xda49 bytes) 2024-09-18T00:58:34.661 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.686 INFO:teuthology.orchestra.run.smithi042.stdout:2909 write 0x86d0849 thru 0x86d333e (0x2af6 bytes) 2024-09-18T00:58:34.689 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:34.681+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:34.711 INFO:teuthology.orchestra.run.smithi042.stdout:2910 write 0xd1a85ae thru 0xd1b4cee (0xc741 bytes) 2024-09-18T00:58:34.721 INFO:teuthology.orchestra.run.smithi042.stdout:2911 read 0x44e847 thru 0x45e5bf (0xfd79 bytes) 2024-09-18T00:58:34.743 INFO:teuthology.orchestra.run.smithi042.stdout:2912 trunc from 0xd1b4cef to 0x176f66c 2024-09-18T00:58:34.786 INFO:teuthology.orchestra.run.smithi042.stdout:2913 trunc from 0x176f66c to 0xe93d334 2024-09-18T00:58:34.790 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:34.785+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:34.797 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.797 INFO:teuthology.orchestra.run.smithi042.stdout:2915 write 0xc384b1a thru 0xc38e25f (0x9746 bytes) 2024-09-18T00:58:34.806 INFO:teuthology.orchestra.run.smithi042.stdout:2916 write 0x88ad32f thru 0x88b3d25 (0x69f7 bytes) 2024-09-18T00:58:34.816 INFO:teuthology.orchestra.run.smithi042.stdout:2917 punch from 0x6a784fb to 0x6a7d738, (0x523d bytes) 2024-09-18T00:58:34.816 INFO:teuthology.orchestra.run.smithi042.stdout:2918 punch from 0x2f612d7 to 0x2f655b2, (0x42db bytes) 2024-09-18T00:58:34.821 INFO:teuthology.orchestra.run.smithi042.stdout:2919 trunc from 0xe93d334 to 0xbbc1993 2024-09-18T00:58:34.824 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.745271698000003, 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.744342107000001, 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:58:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:58:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:58:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:58:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:58:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:58:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:58:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.744328047000003, 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:58:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.743037471999997, 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:58:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:58:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:58:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:58:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:58:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:58:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:58:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:58:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:58:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.743017000000002, 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:58:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:58:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.740222547000002, 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:58:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:58:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.740207363000003, 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:58:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.740184417000002, 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.736006777999997, 2024-09-18T00:58:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:58:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:34.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:58:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:58:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.735985561, 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:58:34.865 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:58:34.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.735964178000003, 2024-09-18T00:58:34.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:58:34.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:58:34.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.732064324, 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.874 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:34.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.732050123, 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:58:34.878 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:58:34.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.879 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:58:34.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:58:34.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.732032085999997, 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.732018019999998, 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:58:34.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.888 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:34.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:58:34.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:58:34.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.727992669000002, 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:58:34.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:58:34.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.727978657000001, 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.896 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:58:34.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:58:34.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:58:34.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:58:34.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.900 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.727964692999997, 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.901 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:58:34.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:58:34.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:58:34.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:58:34.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:58:34.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:34.905 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.966106738999997, 2024-09-18T00:58:34.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:58:34.906 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:34.906 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:58:34.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:34.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:58:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.840630367999999, 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:34.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:34.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:34.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:34.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:34.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:34.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:58:34.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.916 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:34.917 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:34.917 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:34.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:34.919 INFO:teuthology.orchestra.run.smithi042.stdout:2920 trunc from 0xbbc1993 to 0x8e23e96 2024-09-18T00:58:34.919 INFO:teuthology.orchestra.run.smithi042.stdout:2922 trunc from 0x8e23e96 to 0x791bee8 2024-09-18T00:58:34.919 INFO:teuthology.orchestra.run.smithi042.stdout:2923 read 0x5c494dd thru 0x5c4f19c (0x5cc0 bytes) 2024-09-18T00:58:34.919 INFO:teuthology.orchestra.run.smithi042.stdout:2928 write 0x8bb8710 thru 0x8bba1da (0x1acb bytes) 2024-09-18T00:58:34.920 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:34.913+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:34.945 INFO:teuthology.orchestra.run.smithi042.stdout:2929 writesame 0xe377d1a thru 0xe37c8be data_size 0x5(0x4ba5 bytes) 2024-09-18T00:58:34.946 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:34.946 INFO:teuthology.orchestra.run.smithi042.stdout:2932 read 0x401e107 thru 0x40205cd (0x24c7 bytes) 2024-09-18T00:58:34.946 INFO:teuthology.orchestra.run.smithi042.stdout:2933 write 0x5343452 thru 0x534907b (0x5c2a bytes) 2024-09-18T00:58:34.954 INFO:teuthology.orchestra.run.smithi042.stdout:2935 punch from 0xcc8abe0 to 0xcc8b684, (0xaa4 bytes) 2024-09-18T00:58:34.954 INFO:teuthology.orchestra.run.smithi042.stdout:2936 write 0x21f6b60 thru 0x21fc4da (0x597b bytes) 2024-09-18T00:58:34.979 INFO:teuthology.orchestra.run.smithi042.stdout:2937 trunc from 0xe37c8bf to 0x63adc05 2024-09-18T00:58:35.004 INFO:teuthology.orchestra.run.smithi042.stdout:2938 punch from 0x163f26c to 0x16419a9, (0x273d bytes) 2024-09-18T00:58:35.004 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.012 INFO:teuthology.orchestra.run.smithi042.stdout:2941 trunc from 0x63adc05 to 0x1c4f41 2024-09-18T00:58:35.019 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:35.013+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:35.079 INFO:teuthology.orchestra.run.smithi042.stdout:2942 write 0xc84d796 thru 0xc859be4 (0xc44f bytes) 2024-09-18T00:58:35.092 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:35.092 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:35.092 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:35.092 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:35.103 INFO:teuthology.orchestra.run.smithi042.stdout:2943 trunc from 0xc859be5 to 0x47990d2 2024-09-18T00:58:35.113 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:35.125 INFO:teuthology.orchestra.run.smithi042.stdout:2944 writesame 0x607124 thru 0x607186 data_size 0x3(0x63 bytes) 2024-09-18T00:58:35.125 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:35.117+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:35.130 INFO:teuthology.orchestra.run.smithi042.stdout:2945 writesame 0x301937e thru 0x3020da7 data_size 0x13(0x7a2a bytes) 2024-09-18T00:58:35.150 INFO:teuthology.orchestra.run.smithi042.stdout:2946 write 0x58e0c34 thru 0x58ebdf2 (0xb1bf bytes) 2024-09-18T00:58:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:2947 read 0x38a3d78 thru 0x38a8f48 (0x51d1 bytes) 2024-09-18T00:58:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:2950 read 0x422c546 thru 0x4234847 (0x8302 bytes) 2024-09-18T00:58:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:2951 read 0x28a9f70 thru 0x28ac3e9 (0x247a bytes) 2024-09-18T00:58:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:2952 write 0x10fa912 thru 0x1107b1c (0xd20b bytes) 2024-09-18T00:58:35.173 INFO:teuthology.orchestra.run.smithi042.stdout:2953 trunc from 0x58ebdf3 to 0x928f5f3 2024-09-18T00:58:35.182 INFO:teuthology.orchestra.run.smithi042.stdout:2954 punch from 0x8d731ae to 0x8d77446, (0x4298 bytes) 2024-09-18T00:58:35.182 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.182 INFO:teuthology.orchestra.run.smithi042.stdout:2957 write 0x3657fc thru 0x36b181 (0x5986 bytes) 2024-09-18T00:58:35.186 INFO:teuthology.orchestra.run.smithi042.stdout:2958 read 0x38c85be thru 0x38c8fcd (0xa10 bytes) 2024-09-18T00:58:35.186 INFO:teuthology.orchestra.run.smithi042.stdout:2959 punch from 0x8850037 to 0x88583ae, (0x8377 bytes) 2024-09-18T00:58:35.186 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.198 INFO:teuthology.orchestra.run.smithi042.stdout:2962 trunc from 0x928f5f3 to 0x1860f63 2024-09-18T00:58:35.220 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:35.213+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:35.246 INFO:teuthology.orchestra.run.smithi042.stdout:2963 write 0x9168fc8 thru 0x9170cc4 (0x7cfd bytes) 2024-09-18T00:58:35.255 INFO:teuthology.orchestra.run.smithi042.stdout:2966 punch from 0x254639a to 0x254c6f6, (0x635c bytes) 2024-09-18T00:58:35.255 INFO:teuthology.orchestra.run.smithi042.stdout:2967 write 0x80adee thru 0x813474 (0x8687 bytes) 2024-09-18T00:58:35.257 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:35.257 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:35.258 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:35.264 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.264 INFO:teuthology.orchestra.run.smithi042.stdout:2970 read 0x7644bfc thru 0x7654a41 (0xfe46 bytes) 2024-09-18T00:58:35.264 INFO:teuthology.orchestra.run.smithi042.stdout:2971 write 0x4771d49 thru 0x477b60d (0x98c5 bytes) 2024-09-18T00:58:35.270 INFO:teuthology.orchestra.run.smithi042.stdout:2973 write 0x13f2208 thru 0x13f8eed (0x6ce6 bytes) 2024-09-18T00:58:35.274 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:35.277 INFO:teuthology.orchestra.run.smithi042.stdout:2974 read 0x5eddfc1 thru 0x5ee1cf7 (0x3d37 bytes) 2024-09-18T00:58:35.278 INFO:teuthology.orchestra.run.smithi042.stdout:2975 punch from 0x26114d5 to 0x2613f07, (0x2a32 bytes) 2024-09-18T00:58:35.278 INFO:teuthology.orchestra.run.smithi042.stdout:2977 write 0x4a440ee thru 0x4a48fdd (0x4ef0 bytes) 2024-09-18T00:58:35.285 INFO:teuthology.orchestra.run.smithi042.stdout:2978 write 0x29b9479 thru 0x29c9238 (0xfdc0 bytes) 2024-09-18T00:58:35.294 INFO:teuthology.orchestra.run.smithi042.stdout:2979 punch from 0x26c56cf to 0x26d25ed, (0xcf1e bytes) 2024-09-18T00:58:35.307 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:58:35.307 INFO:teuthology.orchestra.run.smithi042.stdout:2981 writesame 0xb89756e thru 0xb89f072 data_size 0xb(0x7b05 bytes) 2024-09-18T00:58:35.307 INFO:teuthology.orchestra.run.smithi042.stdout:2983 read 0x78089f6 thru 0x7813d27 (0xb332 bytes) 2024-09-18T00:58:35.308 INFO:teuthology.orchestra.run.smithi042.stdout:2984 write 0x996cbee thru 0x996ea7d (0x1e90 bytes) 2024-09-18T00:58:35.314 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.314 INFO:teuthology.orchestra.run.smithi042.stdout:2990 writesame 0x1c56f3 thru 0x1cbe78 data_size 0x15(0x6786 bytes) 2024-09-18T00:58:35.317 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.317 INFO:teuthology.orchestra.run.smithi042.stdout:2992 writesame 0xa21490f thru 0xa2203ad data_size 0x69(0xba9f bytes) 2024-09-18T00:58:35.321 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:35.313+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:35.326 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.329 INFO:teuthology.orchestra.run.smithi042.stdout:2995 trunc from 0xb89f073 to 0xa2f10be 2024-09-18T00:58:35.340 INFO:teuthology.orchestra.run.smithi042.stdout:2996 punch from 0x3e6cf38 to 0x3e789ba, (0xba82 bytes) 2024-09-18T00:58:35.357 INFO:teuthology.orchestra.run.smithi042.stdout:2997 writesame 0xe716336 thru 0xe71c39d data_size 0x14(0x6068 bytes) 2024-09-18T00:58:35.357 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.371 INFO:teuthology.orchestra.run.smithi042.stdout:3000 trunc from 0xe71c39e to 0x603013e 2024-09-18T00:58:35.390 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.390 INFO:teuthology.orchestra.run.smithi042.stdout:3003 writesame 0x3f4e382 thru 0x3f5649e data_size 0x1(0x811d bytes) 2024-09-18T00:58:35.422 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:35.417+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:35.427 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:35.427 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:35.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:35.427 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:35.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.427 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:58:35.427 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:35.427 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 68.110754717000006, 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101264911, 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4226", 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2155402750", 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313760+0000", 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.612e-06 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313768+0000", 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9249999999999995e-06 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313786+0000", 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8253999999999998e-05 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313803+0000", 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313935+0000", 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013180399999999999 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313954+0000", 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8768999999999999e-05 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.411874+0000", 2024-09-18T00:58:35.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.097920646 2024-09-18T00:58:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412292+0000", 2024-09-18T00:58:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041777699999999998 2024-09-18T00:58:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:35.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412675+0000", 2024-09-18T00:58:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038262300000000001 2024-09-18T00:58:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414019+0000", 2024-09-18T00:58:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013436699999999999 2024-09-18T00:58:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414872+0000", 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085320999999999997 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414955+0000", 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3405999999999994e-05 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.415023+0000", 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7825999999999997e-05 2024-09-18T00:58:35.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15500 3.11 3:8ed6a97b:::rbd_data.117c53e529c1.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.609994696999998, 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046818956000000002, 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15500 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814521+0000", 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.977e-06 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814536+0000", 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.487e-05 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814572+0000", 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5567e-05 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814588+0000", 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6546e-05 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814675+0000", 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6675999999999995e-05 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814839+0000", 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016414199999999999 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.819908+0000", 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050695280000000002 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861139+0000", 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041230771999999999 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861208+0000", 2024-09-18T00:58:35.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8722999999999999e-05 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861277+0000", 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9246999999999996e-05 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861337+0000", 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9908000000000003e-05 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26860 3.a 3:523ff797:::rbd_data.117c4c7db1a0.0000000000000039:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.051820578000001, 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028475341000000001, 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26860 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.441 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:35.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:35.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372693+0000", 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0550000000000001e-06 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372699+0000", 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372714+0000", 2024-09-18T00:58:35.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4673e-05 2024-09-18T00:58:35.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372722+0000", 2024-09-18T00:58:35.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4659999999999999e-06 2024-09-18T00:58:35.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372787+0000", 2024-09-18T00:58:35.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5288000000000002e-05 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372874+0000", 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6485999999999998e-05 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.377992+0000", 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051177159999999996 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401008+0000", 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02301603 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401026+0000", 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712e-05 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401082+0000", 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5648e-05 2024-09-18T00:58:35.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401168+0000", 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5476999999999995e-05 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64289 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 3227648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8381308829999998, 2024-09-18T00:58:35.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025443502999999999, 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64289 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.84e-07 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586385+0000", 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2060000000000001e-06 2024-09-18T00:58:35.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586392+0000", 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586396+0000", 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5859999999999998e-06 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611580+0000", 2024-09-18T00:58:35.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025183646000000001 2024-09-18T00:58:35.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611667+0000", 2024-09-18T00:58:35.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7350999999999998e-05 2024-09-18T00:58:35.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611825+0000", 2024-09-18T00:58:35.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015804999999999999 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64403 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 475136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8371843999999999, 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025296714000000001, 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.453 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64403 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587329+0000", 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587331+0000", 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9889999999999999e-06 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587338+0000", 2024-09-18T00:58:35.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5209999999999997e-06 2024-09-18T00:58:35.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587343+0000", 2024-09-18T00:58:35.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:35.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612401+0000", 2024-09-18T00:58:35.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025057505000000001 2024-09-18T00:58:35.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612478+0000", 2024-09-18T00:58:35.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6817999999999995e-05 2024-09-18T00:58:35.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612625+0000", 2024-09-18T00:58:35.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014740099999999999 2024-09-18T00:58:35.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.459 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.459 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64404 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 655360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.460 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:35.460 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8371631130000008, 2024-09-18T00:58:35.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025540833999999998, 2024-09-18T00:58:35.460 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64404 2024-09-18T00:58:35.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.461 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:35.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:35.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:35.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:35.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587352+0000", 2024-09-18T00:58:35.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-18T00:58:35.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587362+0000", 2024-09-18T00:58:35.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0125e-05 2024-09-18T00:58:35.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587367+0000", 2024-09-18T00:58:35.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1150000000000002e-06 2024-09-18T00:58:35.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612671+0000", 2024-09-18T00:58:35.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025303638 2024-09-18T00:58:35.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612751+0000", 2024-09-18T00:58:35.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0107999999999999e-05 2024-09-18T00:58:35.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612891+0000", 2024-09-18T00:58:35.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013946 2024-09-18T00:58:35.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.469 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.470 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64405 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.470 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:35.470 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8371377580000008, 2024-09-18T00:58:35.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025814888000000001, 2024-09-18T00:58:35.470 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64405 2024-09-18T00:58:35.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.471 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:35.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:35.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:35.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:35.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587379+0000", 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1439999999999999e-06 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587398+0000", 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587405+0000", 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3740000000000003e-06 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612931+0000", 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025526604000000001 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613008+0000", 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6508000000000007e-05 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613190+0000", 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018205200000000001 2024-09-18T00:58:35.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64406 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8370857580000006, 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026015033, 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64406 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587428+0000", 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0399999999999996e-07 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587431+0000", 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1870000000000001e-06 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587441+0000", 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0349999999999999e-05 2024-09-18T00:58:35.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587447+0000", 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.083e-06 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613236+0000", 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025788953 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613288+0000", 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2275999999999999e-05 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613442+0000", 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015358 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64407 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:35.480 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8370567589999993, 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026326130999999999, 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64407 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587459+0000", 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3750000000000001e-06 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587465+0000", 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7150000000000003e-06 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587469+0000", 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8060000000000002e-06 2024-09-18T00:58:35.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613488+0000", 2024-09-18T00:58:35.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026018645999999999 2024-09-18T00:58:35.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613576+0000", 2024-09-18T00:58:35.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7892000000000005e-05 2024-09-18T00:58:35.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613782+0000", 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000206233 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64408 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8370365159999995, 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026568468000000001, 2024-09-18T00:58:35.486 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64408 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587477+0000", 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7500000000000001e-07 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587479+0000", 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1900000000000002e-06 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587487+0000", 2024-09-18T00:58:35.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1869999999999997e-06 2024-09-18T00:58:35.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587492+0000", 2024-09-18T00:58:35.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3149999999999999e-06 2024-09-18T00:58:35.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613823+0000", 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026330339000000001 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613885+0000", 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2391999999999995e-05 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614045+0000", 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015966999999999999 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.490 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64409 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8370135390000009, 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026785499000000001, 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.491 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64409 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587500+0000", 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587502+0000", 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9420000000000002e-06 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587508+0000", 2024-09-18T00:58:35.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1840000000000004e-06 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587513+0000", 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9640000000000002e-06 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614075+0000", 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026561889000000002 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614124+0000", 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9177e-05 2024-09-18T00:58:35.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614285+0000", 2024-09-18T00:58:35.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016103100000000001 2024-09-18T00:58:35.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.495 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.495 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64410 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8369935900000005, 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026987559000000001, 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64410 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587520+0000", 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587522+0000", 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1500000000000002e-06 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587529+0000", 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5839999999999997e-06 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587534+0000", 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.826e-06 2024-09-18T00:58:35.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614317+0000", 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026783065000000002 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614385+0000", 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7682999999999996e-05 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614507+0000", 2024-09-18T00:58:35.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121859 2024-09-18T00:58:35.500 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.500 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.500 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.500 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64411 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.500 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:35.500 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8369721049999992, 2024-09-18T00:58:35.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027224245000000001, 2024-09-18T00:58:35.500 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64411 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587543+0000", 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:58:35.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587550+0000", 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5810000000000002e-06 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587555+0000", 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0490000000000003e-06 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614543+0000", 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026988056 2024-09-18T00:58:35.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614619+0000", 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5932000000000004e-05 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614765+0000", 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014603899999999999 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64412 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2097152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8369513600000005, 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027478605, 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64412 2024-09-18T00:58:35.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.506 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:35.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587562+0000", 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6499999999999999e-07 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587564+0000", 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3300000000000001e-06 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587571+0000", 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2949999999999999e-06 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587575+0000", 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6360000000000002e-06 2024-09-18T00:58:35.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614798+0000", 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027222860000000002 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614874+0000", 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5383000000000007e-05 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615040+0000", 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016653600000000001 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64413 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8369305560000004, 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027709421000000001, 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64413 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587583+0000", 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587585+0000", 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-18T00:58:35.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587592+0000", 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9700000000000002e-06 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587597+0000", 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8289999999999998e-06 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615071+0000", 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027474218000000002 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615143+0000", 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1505000000000001e-05 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615292+0000", 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014900199999999999 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64414 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2457600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8369090180000001, 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027920637000000002, 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64414 2024-09-18T00:58:35.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:35.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.08e-07 2024-09-18T00:58:35.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587606+0000", 2024-09-18T00:58:35.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1299999999999999e-06 2024-09-18T00:58:35.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587612+0000", 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1319999999999999e-06 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587618+0000", 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.147e-06 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615323+0000", 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027705423 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615366+0000", 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2914000000000003e-05 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615524+0000", 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015848300000000001 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64415 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8368846970000003, 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028241265000000002, 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64415 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:35.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:35.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587631+0000", 2024-09-18T00:58:35.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5550000000000001e-06 2024-09-18T00:58:35.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587639+0000", 2024-09-18T00:58:35.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.807e-06 2024-09-18T00:58:35.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587644+0000", 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2630000000000003e-06 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615633+0000", 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027989342 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615703+0000", 2024-09-18T00:58:35.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9956000000000006e-05 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615869+0000", 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016597500000000001 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64416 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2818048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8368614000000001, 2024-09-18T00:58:35.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028575605, 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64416 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:35.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587652+0000", 2024-09-18T00:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-18T00:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587655+0000", 2024-09-18T00:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-18T00:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587662+0000", 2024-09-18T00:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7789999999999996e-06 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587668+0000", 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2109999999999997e-06 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615916+0000", 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028248412000000001 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616009+0000", 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2682999999999993e-05 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616227+0000", 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218345 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64417 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8368373889999994, 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028891495, 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64417 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587676+0000", 2024-09-18T00:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.34e-07 2024-09-18T00:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587683+0000", 2024-09-18T00:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8739999999999997e-06 2024-09-18T00:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587692+0000", 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2159999999999995e-06 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587698+0000", 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.716e-06 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616259+0000", 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028561733999999998 2024-09-18T00:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616369+0000", 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110104 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616567+0000", 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000197417 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64418 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.8368069449999993, 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029401729000000001, 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64418 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8400000000000004e-07 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587709+0000", 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5390000000000002e-06 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587717+0000", 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0819999999999999e-06 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587722+0000", 2024-09-18T00:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4140000000000002e-06 2024-09-18T00:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616869+0000", 2024-09-18T00:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029146763999999999 2024-09-18T00:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616926+0000", 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7250000000000002e-05 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.617108+0000", 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181096 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout:3004 writesame 0x319818c thru 0x3198525 data_size 0x1(0x39a bytes) 2024-09-18T00:58:35.541 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout:3006 read 0x2f9fe9e thru 0x2facae9 (0xcc4c bytes) 2024-09-18T00:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout:3007 writesame 0x14f9547 thru 0x14fd819 data_size 0x1(0x42d3 bytes) 2024-09-18T00:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout:3009 read 0x2c6c2a6 thru 0x2c78086 (0xbde1 bytes) 2024-09-18T00:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout:3010 read 0x5ca6ac6 thru 0x5cae963 (0x7e9e bytes) 2024-09-18T00:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout:3011 trunc from 0x603013e to 0xe3ce727 2024-09-18T00:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout:3012 punch from 0x3a31e8c to 0x3a355ba, (0x372e bytes) 2024-09-18T00:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout:3015 read 0x1f45384 thru 0x1f52876 (0xd4f3 bytes) 2024-09-18T00:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout:3016 trunc from 0xe3ce727 to 0xa61437b 2024-09-18T00:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout:3017 write 0x8ab7651 thru 0x8ab8f39 (0x18e9 bytes) 2024-09-18T00:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout:3018 write 0xa100508 thru 0xa10770f (0x7208 bytes) 2024-09-18T00:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout:3019 trunc from 0xa61437b to 0x7b914c8 2024-09-18T00:58:35.543 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:35.533+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout:3020 write 0xd2f54f6 thru 0xd303fba (0xeac5 bytes) 2024-09-18T00:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout:3022 writesame 0x5515c35 thru 0x551ec89 data_size 0x1(0x9055 bytes) 2024-09-18T00:58:35.613 INFO:teuthology.orchestra.run.smithi042.stdout:3023 punch from 0x2f56006 to 0x2f56259, (0x253 bytes) 2024-09-18T00:58:35.613 INFO:teuthology.orchestra.run.smithi042.stdout:3026 punch from 0x1b77415 to 0x1b7b0c3, (0x3cae bytes) 2024-09-18T00:58:35.614 INFO:teuthology.orchestra.run.smithi042.stdout:3027 read 0x22d352f thru 0x22d75a4 (0x4076 bytes) 2024-09-18T00:58:35.614 INFO:teuthology.orchestra.run.smithi042.stdout:3030 write 0x4c1789e thru 0x4c240d3 (0xc836 bytes) 2024-09-18T00:58:35.621 INFO:teuthology.orchestra.run.smithi042.stdout:3031 read 0x523e412 thru 0x524a53d (0xc12c bytes) 2024-09-18T00:58:35.623 INFO:teuthology.orchestra.run.smithi042.stdout:3032 trunc from 0xd303fbb to 0xc6e06fc 2024-09-18T00:58:35.642 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:35.637+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:35.645 INFO:teuthology.orchestra.run.smithi042.stdout:3033 punch from 0x6756ec8 to 0x6759cfa, (0x2e32 bytes) 2024-09-18T00:58:35.645 INFO:teuthology.orchestra.run.smithi042.stdout:3034 read 0x484d6eb thru 0x4859fd8 (0xc8ee bytes) 2024-09-18T00:58:35.646 INFO:teuthology.orchestra.run.smithi042.stdout:3036 write 0x3e28c9c thru 0x3e309ab (0x7d10 bytes) 2024-09-18T00:58:35.653 INFO:teuthology.orchestra.run.smithi042.stdout:3037 write 0x3ce1266 thru 0x3cefa12 (0xe7ad bytes) 2024-09-18T00:58:35.658 INFO:teuthology.orchestra.run.smithi042.stdout:3038 punch from 0x37cf527 to 0x37d7f04, (0x89dd bytes) 2024-09-18T00:58:35.658 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.658 INFO:teuthology.orchestra.run.smithi042.stdout:3041 write 0x5cab2b7 thru 0x5cad089 (0x1dd3 bytes) 2024-09-18T00:58:35.663 INFO:teuthology.orchestra.run.smithi042.stdout:3042 read 0x137d164 thru 0x138b78f (0xe62c bytes) 2024-09-18T00:58:35.663 INFO:teuthology.orchestra.run.smithi042.stdout:3043 punch from 0x8c7e78e to 0x8c8e541, (0xfdb3 bytes) 2024-09-18T00:58:35.663 INFO:teuthology.orchestra.run.smithi042.stdout:3044 read 0x2a4057b thru 0x2a45255 (0x4cdb bytes) 2024-09-18T00:58:35.663 INFO:teuthology.orchestra.run.smithi042.stdout:3045 read 0x5d70d2e thru 0x5d7abad (0x9e80 bytes) 2024-09-18T00:58:35.663 INFO:teuthology.orchestra.run.smithi042.stdout:3047 read 0x74915cc thru 0x7492e76 (0x18ab bytes) 2024-09-18T00:58:35.669 INFO:teuthology.orchestra.run.smithi042.stdout:3048 trunc from 0xc6e06fc to 0x86a9916 2024-09-18T00:58:35.679 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.679 INFO:teuthology.orchestra.run.smithi042.stdout:3050 writesame 0x53b7bb6 thru 0x53bca96 data_size 0x35(0x4ee1 bytes) 2024-09-18T00:58:35.684 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:35.684 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:35.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:35.684 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:71485 3.1b 3:da31ff53:::rbd_data.117c8d899572.00000000000000a9:head [set-alloc-hint object_size 524288 write_size 524288,write-same in=53b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T00:58:35.684 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:35.682753+0000", 2024-09-18T00:58:35.684 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.0012516579999999999, 2024-09-18T00:58:35.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013277040000000001, 2024-09-18T00:58:35.684 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:35.684 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:35.684 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "waiting for sub ops", 2024-09-18T00:58:35.684 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:35.685 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:35.685 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:35.685 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 71485 2024-09-18T00:58:35.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:35.685 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:35.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:35.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:35.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:35.682753+0000", 2024-09-18T00:58:35.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:35.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:35.682753+0000", 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:35.682755+0000", 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9039999999999999e-06 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:35.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:35.682766+0000", 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.044e-05 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:35.682795+0000", 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9235999999999999e-05 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:35.682812+0000", 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7122999999999999e-05 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:35.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:35.682894+0000", 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1866000000000005e-05 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:35.683066+0000", 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017227099999999999 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:35.683259+0000", 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019290099999999999 2024-09-18T00:58:35.688 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:35.689 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:35.689 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:35.689 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:35.689 INFO:teuthology.orchestra.run.smithi195.stdout: ], 2024-09-18T00:58:35.689 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 1 2024-09-18T00:58:35.689 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:35.690 INFO:teuthology.orchestra.run.smithi042.stdout:3052 read 0x2455105 thru 0x2460bd8 (0xbad4 bytes) 2024-09-18T00:58:35.690 INFO:teuthology.orchestra.run.smithi042.stdout:3055 read 0x635f8a5 thru 0x636a4a7 (0xac03 bytes) 2024-09-18T00:58:35.690 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.690 INFO:teuthology.orchestra.run.smithi042.stdout:3057 punch from 0x6f2b776 to 0x6f33083, (0x790d bytes) 2024-09-18T00:58:35.690 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.690 INFO:teuthology.orchestra.run.smithi042.stdout:3059 read 0x33fe2f3 thru 0x3403a39 (0x5747 bytes) 2024-09-18T00:58:35.698 INFO:teuthology.orchestra.run.smithi042.stdout:3060 write 0xa2f8ad1 thru 0xa2fcdde (0x430e bytes) 2024-09-18T00:58:35.699 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:35.705 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:58:35.727 INFO:teuthology.orchestra.run.smithi042.stdout:3061 write 0xc88c299 thru 0xc899749 (0xd4b1 bytes) 2024-09-18T00:58:35.733 INFO:teuthology.orchestra.run.smithi042.stdout:3062 punch from 0x4018902 to 0x40258af, (0xcfad bytes) 2024-09-18T00:58:35.733 INFO:teuthology.orchestra.run.smithi042.stdout:3064 write 0xc727802 thru 0xc729442 (0x1c41 bytes) 2024-09-18T00:58:35.737 INFO:teuthology.orchestra.run.smithi042.stdout:3065 writesame 0x828cb8d thru 0x829af86 data_size 0x8e(0xe3fa bytes) 2024-09-18T00:58:35.742 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:35.737+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:35.743 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.744 INFO:teuthology.orchestra.run.smithi042.stdout:3067 read 0xc7af89 thru 0xc7eea1 (0x3f19 bytes) 2024-09-18T00:58:35.744 INFO:teuthology.orchestra.run.smithi042.stdout:3068 write 0x6653c94 thru 0x665e1ac (0xa519 bytes) 2024-09-18T00:58:35.749 INFO:teuthology.orchestra.run.smithi042.stdout:3069 writesame 0x36d536b thru 0x36d8419 data_size 0xb(0x30af bytes) 2024-09-18T00:58:35.756 INFO:teuthology.orchestra.run.smithi042.stdout:3070 punch from 0x182e2ec to 0x18356da, (0x73ee bytes) 2024-09-18T00:58:35.756 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.756 INFO:teuthology.orchestra.run.smithi042.stdout:3072 punch from 0x1b864e7 to 0x1b9119b, (0xacb4 bytes) 2024-09-18T00:58:35.756 INFO:teuthology.orchestra.run.smithi042.stdout:3073 write 0x220c9b8 thru 0x221b773 (0xedbc bytes) 2024-09-18T00:58:35.763 INFO:teuthology.orchestra.run.smithi042.stdout:3074 read 0xc5bf6e7 thru 0xc5bfb4f (0x469 bytes) 2024-09-18T00:58:35.763 INFO:teuthology.orchestra.run.smithi042.stdout:3075 write 0xa425a13 thru 0xa433e2b (0xe419 bytes) 2024-09-18T00:58:35.771 INFO:teuthology.orchestra.run.smithi042.stdout:3077 read 0x8885650 thru 0x8892ae2 (0xd493 bytes) 2024-09-18T00:58:35.771 INFO:teuthology.orchestra.run.smithi042.stdout:3078 write 0xacdca01 thru 0xacec8a6 (0xfea6 bytes) 2024-09-18T00:58:35.778 INFO:teuthology.orchestra.run.smithi042.stdout:3079 read 0xbf716fd thru 0xbf81318 (0xfc1c bytes) 2024-09-18T00:58:35.778 INFO:teuthology.orchestra.run.smithi042.stdout:3080 read 0xc158fe8 thru 0xc165ed0 (0xcee9 bytes) 2024-09-18T00:58:35.808 INFO:teuthology.orchestra.run.smithi042.stdout:3081 trunc from 0xc89974a to 0x1346464 2024-09-18T00:58:35.843 INFO:teuthology.orchestra.run.smithi042.stdout:3084 read 0x1e5d23 thru 0x1f024b (0xa529 bytes) 2024-09-18T00:58:35.843 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:35.837+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:35.843 INFO:teuthology.orchestra.run.smithi042.stdout:3085 read 0xe30c93 thru 0xe404e9 (0xf857 bytes) 2024-09-18T00:58:35.856 INFO:teuthology.orchestra.run.smithi042.stdout:3086 writesame 0x1d07730 thru 0x1d16508 data_size 0x1(0xedd9 bytes) 2024-09-18T00:58:35.868 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:35.868 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:35.868 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:35.868 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:35.869 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:35.882 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:35.899 INFO:teuthology.orchestra.run.smithi042.stdout:3087 write 0xd98952d thru 0xd994a61 (0xb535 bytes) 2024-09-18T00:58:35.905 INFO:teuthology.orchestra.run.smithi042.stdout:3088 punch from 0x7d636cd to 0x7d71f0a, (0xe83d bytes) 2024-09-18T00:58:35.919 INFO:teuthology.orchestra.run.smithi042.stdout:3089 trunc from 0xd994a62 to 0x88d5e00 2024-09-18T00:58:35.939 INFO:teuthology.orchestra.run.smithi042.stdout:3090 writesame 0x5394328 thru 0x539f144 data_size 0x1d(0xae1d bytes) 2024-09-18T00:58:35.944 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:35.937+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:35.946 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.959 INFO:teuthology.orchestra.run.smithi042.stdout:3092 writesame 0x8e9e81d thru 0x8eaaf48 data_size 0x2a(0xc72c bytes) 2024-09-18T00:58:35.959 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:35.959 INFO:teuthology.orchestra.run.smithi042.stdout:3097 punch from 0x866b624 to 0x8677a6b, (0xc447 bytes) 2024-09-18T00:58:35.959 INFO:teuthology.orchestra.run.smithi042.stdout:3098 writesame 0x3b6f780 thru 0x3b761bb data_size 0x1a(0x6a3c bytes) 2024-09-18T00:58:35.977 INFO:teuthology.orchestra.run.smithi042.stdout:3099 trunc from 0x8eaaf49 to 0x4aced43 2024-09-18T00:58:36.004 INFO:teuthology.orchestra.run.smithi042.stdout:3100 read 0x1d42fde thru 0x1d50507 (0xd52a bytes) 2024-09-18T00:58:36.017 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:36.017 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:36.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:36.017 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:36.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.017 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1670 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1335296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.017 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.847550626, 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083399740999999999, 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1670 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:36.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:36.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166573+0000", 2024-09-18T00:58:36.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1829999999999999e-06 2024-09-18T00:58:36.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166578+0000", 2024-09-18T00:58:36.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0030000000000002e-06 2024-09-18T00:58:36.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166594+0000", 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6470999999999999e-05 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166607+0000", 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2831e-05 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249706+0000", 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083099022999999994 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249843+0000", 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136518 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249971+0000", 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012871200000000001 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.022 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1671 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1376256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.847499849000002, 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083700574, 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1671 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166623+0000", 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.066e-06 2024-09-18T00:58:36.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166641+0000", 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7728e-05 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166659+0000", 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7850999999999999e-05 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166672+0000", 2024-09-18T00:58:36.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3020000000000001e-05 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250034+0000", 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083362085000000002 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250145+0000", 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111045 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250323+0000", 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017777900000000001 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1672 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.847426390000003, 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083853480999999994, 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.027 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.028 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1672 2024-09-18T00:58:36.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.028 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:36.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166697+0000", 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0920000000000001e-06 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166702+0000", 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0799999999999996e-06 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166718+0000", 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6167999999999998e-05 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166745+0000", 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7177000000000001e-05 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250382+0000", 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083636792000000001 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250476+0000", 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250549+0000", 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3282999999999996e-05 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.031 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1673 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.847359281999999, 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084151880999999998, 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1673 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166764+0000", 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166770+0000", 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5969999999999999e-06 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166786+0000", 2024-09-18T00:58:36.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6439999999999998e-05 2024-09-18T00:58:36.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166799+0000", 2024-09-18T00:58:36.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2934e-05 2024-09-18T00:58:36.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250605+0000", 2024-09-18T00:58:36.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083806357999999997 2024-09-18T00:58:36.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250724+0000", 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011821500000000001 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250915+0000", 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000191251 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.036 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1674 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.847259696999998, 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084239101999999996, 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1674 2024-09-18T00:58:36.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166864+0000", 2024-09-18T00:58:36.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3629999999999999e-06 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166883+0000", 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9083e-05 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166902+0000", 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8711000000000001e-05 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166915+0000", 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3375e-05 2024-09-18T00:58:36.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250962+0000", 2024-09-18T00:58:36.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084047254000000002 2024-09-18T00:58:36.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251029+0000", 2024-09-18T00:58:36.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6315000000000005e-05 2024-09-18T00:58:36.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251102+0000", 2024-09-18T00:58:36.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3001000000000001e-05 2024-09-18T00:58:36.041 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.041 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.041 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.041 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1675 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1540096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.847187845999997, 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084536891000000003, 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1675 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166936+0000", 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2720000000000001e-06 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166941+0000", 2024-09-18T00:58:36.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7370000000000003e-06 2024-09-18T00:58:36.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166958+0000", 2024-09-18T00:58:36.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-18T00:58:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166972+0000", 2024-09-18T00:58:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3569e-05 2024-09-18T00:58:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251159+0000", 2024-09-18T00:58:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084187467000000002 2024-09-18T00:58:36.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251300+0000", 2024-09-18T00:58:36.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014059200000000001 2024-09-18T00:58:36.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251471+0000", 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017168000000000001 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1676 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.847134965999999, 2024-09-18T00:58:36.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084738637000000006, 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1676 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166988+0000", 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166993+0000", 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8339999999999998e-06 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167009+0000", 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6028e-05 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167022+0000", 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2675e-05 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251517+0000", 2024-09-18T00:58:36.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084494975999999999 2024-09-18T00:58:36.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251576+0000", 2024-09-18T00:58:36.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8644999999999998e-05 2024-09-18T00:58:36.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251726+0000", 2024-09-18T00:58:36.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015039 2024-09-18T00:58:36.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.052 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.053 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1677 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.053 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:36.053 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.847083294999997, 2024-09-18T00:58:36.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085055403000000002, 2024-09-18T00:58:36.053 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.054 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.054 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.054 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.054 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.054 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.054 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1677 2024-09-18T00:58:36.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.054 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:36.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:36.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167040+0000", 2024-09-18T00:58:36.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:36.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167046+0000", 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167062+0000", 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6354e-05 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167075+0000", 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2965e-05 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251789+0000", 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084713940000000001 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251916+0000", 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000127176 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252094+0000", 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017833599999999999 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.058 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1678 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.847031532000003, 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085316746999999998, 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1678 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.059 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167092+0000", 2024-09-18T00:58:36.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:58:36.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167097+0000", 2024-09-18T00:58:36.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7029999999999997e-06 2024-09-18T00:58:36.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167113+0000", 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167125+0000", 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2517000000000001e-05 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252157+0000", 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085031933000000004 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252277+0000", 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119708 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252407+0000", 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013031800000000001 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.063 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1679 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.846980813000002, 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085557195000000003, 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1679 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167143+0000", 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:58:36.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167170+0000", 2024-09-18T00:58:36.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7523000000000001e-05 2024-09-18T00:58:36.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167189+0000", 2024-09-18T00:58:36.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8530999999999999e-05 2024-09-18T00:58:36.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167215+0000", 2024-09-18T00:58:36.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6676999999999999e-05 2024-09-18T00:58:36.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252445+0000", 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085229416000000002 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252529+0000", 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4740000000000005e-05 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252699+0000", 2024-09-18T00:58:36.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016920800000000001 2024-09-18T00:58:36.070 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.070 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.070 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.071 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1680 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.071 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:36.071 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.846887356000003, 2024-09-18T00:58:36.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085810110999999994, 2024-09-18T00:58:36.071 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.071 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.072 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.072 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.072 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.072 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.072 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1680 2024-09-18T00:58:36.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.073 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:36.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:36.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167236+0000", 2024-09-18T00:58:36.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.048e-06 2024-09-18T00:58:36.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167242+0000", 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6459999999999998e-06 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167259+0000", 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.696e-05 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167271+0000", 2024-09-18T00:58:36.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.292e-05 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252839+0000", 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085567292000000003 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252924+0000", 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5511000000000002e-05 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253045+0000", 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120734 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1681 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.84678126, 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085887343000000005, 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.078 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1681 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167342+0000", 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167348+0000", 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4519999999999996e-06 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167372+0000", 2024-09-18T00:58:36.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4261999999999999e-05 2024-09-18T00:58:36.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167400+0000", 2024-09-18T00:58:36.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-18T00:58:36.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253091+0000", 2024-09-18T00:58:36.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085690895000000003 2024-09-18T00:58:36.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253154+0000", 2024-09-18T00:58:36.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3262000000000005e-05 2024-09-18T00:58:36.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253228+0000", 2024-09-18T00:58:36.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4318000000000002e-05 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1682 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.846703179999999, 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.086061112999999995, 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.083 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1682 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168114+0000", 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00069532600000000002 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168123+0000", 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8319999999999995e-06 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168147+0000", 2024-09-18T00:58:36.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3536999999999998e-05 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168163+0000", 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6317e-05 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253274+0000", 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085111390999999995 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253362+0000", 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7384000000000002e-05 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253480+0000", 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011832600000000001 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1683 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1867776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.087 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.845926048999999, 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085456638000000001, 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1683 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169938+0000", 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017419079999999999 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169951+0000", 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2534999999999999e-05 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169990+0000", 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9004999999999997e-05 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170011+0000", 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.156e-05 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253534+0000", 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083522759000000002 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253585+0000", 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1316999999999997e-05 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253653+0000", 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7553999999999997e-05 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.091 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1684 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.844076413000003, 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083787188999999998, 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.092 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1684 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170048+0000", 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.674e-06 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170055+0000", 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3549999999999999e-06 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170073+0000", 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7662000000000001e-05 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170087+0000", 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4229e-05 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253701+0000", 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083614346000000006 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253752+0000", 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0760000000000002e-05 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253833+0000", 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1162999999999995e-05 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1685 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.844015616, 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083900785000000005, 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.096 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1685 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:36.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170108+0000", 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.229e-06 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170114+0000", 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0079999999999999e-06 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170132+0000", 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7734e-05 2024-09-18T00:58:36.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170145+0000", 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2977e-05 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253866+0000", 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083721208000000005 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253934+0000", 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.834e-05 2024-09-18T00:58:36.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254007+0000", 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3288999999999996e-05 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1686 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.843960404999997, 2024-09-18T00:58:36.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084083526000000006, 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1686 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170163+0000", 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1909999999999999e-06 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170180+0000", 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7323e-05 2024-09-18T00:58:36.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170198+0000", 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7723e-05 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170212+0000", 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3645999999999999e-05 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254050+0000", 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083838115000000005 2024-09-18T00:58:36.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254136+0000", 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6106999999999994e-05 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254245+0000", 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010942099999999999 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.104 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1687 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.843892103000002, 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084186574, 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.105 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1687 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170647+0000", 2024-09-18T00:58:36.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416649 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170655+0000", 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6510000000000007e-06 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170683+0000", 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7093999999999998e-05 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170701+0000", 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8383e-05 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254283+0000", 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083582091999999997 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254339+0000", 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6187999999999998e-05 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254417+0000", 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7516999999999997e-05 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1688 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.843396388999999, 2024-09-18T00:58:36.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083931026000000006, 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1688 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173553+0000", 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002827333 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173566+0000", 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3276e-05 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173603+0000", 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6177000000000003e-05 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173628+0000", 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5343000000000001e-05 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254460+0000", 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.080831745999999996 2024-09-18T00:58:36.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254547+0000", 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7045000000000004e-05 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254657+0000", 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011010600000000001 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.239259187000002, 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:58:36.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:58:36.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:58:36.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:58:36.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:58:36.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:58:36.119 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.119 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.119 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.119 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.119 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.119 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:36.121 INFO:teuthology.orchestra.run.smithi042.stdout:3101 trunc from 0x4aced43 to 0xadeb15a 2024-09-18T00:58:36.121 INFO:teuthology.orchestra.run.smithi042.stdout:3102 clone 26 order 23 su 131072 sc 15 2024-09-18T00:58:36.121 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:36.122 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:36.113+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:36.221 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:36.217+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:36.249 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:36.249 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:36.249 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:36.249 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:36.263 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:36.303 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:58:36.303 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:58:36.303 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T00:58:36.303 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 1.000000 2024-09-18T00:58:36.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-18T00:58:36.322 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:36.317+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:36.384 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:36.384 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:36.384 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:36.384 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:36.384 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:36.398 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:36.441 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:36.437+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:36.497 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:36.542 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:36.537+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:36.566 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:36.566 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.154509241, 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T00:58:36.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.572 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:26797 3.0 3:03a9420d:::rbd_data.117c4c7db1a0.0000000000000071:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.376082442000001, 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073789632999999993, 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.573 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 26797 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187640+0000", 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187653+0000", 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2357e-05 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187684+0000", 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1582999999999999e-05 2024-09-18T00:58:36.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187699+0000", 2024-09-18T00:58:36.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5003e-05 2024-09-18T00:58:36.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187806+0000", 2024-09-18T00:58:36.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106776 2024-09-18T00:58:36.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.188026+0000", 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021975799999999999 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7", 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.201663+0000", 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013637768 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261226+0000", 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059562916 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:36.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261261+0000", 2024-09-18T00:58:36.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4360000000000003e-05 2024-09-18T00:58:36.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:36.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261354+0000", 2024-09-18T00:58:36.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3041999999999995e-05 2024-09-18T00:58:36.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261427+0000", 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3553000000000005e-05 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64346 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974830856000001, 2024-09-18T00:58:36.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048905196999999997, 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64346 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588890+0000", 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3e-07 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588893+0000", 2024-09-18T00:58:36.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.106e-06 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588904+0000", 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1037e-05 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588911+0000", 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2729999999999997e-06 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637426+0000", 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048514483999999997 2024-09-18T00:58:36.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637537+0000", 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110901 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637795+0000", 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025776599999999998 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64347 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974798982999999, 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049469502999999998, 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64347 2024-09-18T00:58:36.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588922+0000", 2024-09-18T00:58:36.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4300000000000003e-07 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588925+0000", 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588934+0000", 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0329999999999997e-06 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588946+0000", 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1547e-05 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637872+0000", 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048926202000000002 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638047+0000", 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174566 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638391+0000", 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034401499999999998 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64348 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974758113, 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050239487999999999, 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.588 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64348 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:36.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588963+0000", 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.133e-06 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588969+0000", 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4229999999999996e-06 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588996+0000", 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7396e-05 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589163+0000", 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016677900000000001 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638498+0000", 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049334743 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638715+0000", 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000217414 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639202+0000", 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048660000000000001 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64349 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.592 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974529208, 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050607656000000001, 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64349 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589192+0000", 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.536e-06 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589201+0000", 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-06 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589222+0000", 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589241+0000", 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8960000000000001e-05 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639285+0000", 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050043892999999999 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639457+0000", 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017286299999999999 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639799+0000", 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034117400000000002 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64350 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974458863000001, 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051028502000000003, 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.597 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64350 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589262+0000", 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1570000000000001e-06 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589280+0000", 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7943000000000001e-05 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589299+0000", 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8940000000000002e-05 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589316+0000", 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6654e-05 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639872+0000", 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050556151000000001 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639988+0000", 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115423 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640290+0000", 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030223400000000002 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64351 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974383122999999, 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051389975999999997, 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.601 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64351 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589338+0000", 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589345+0000", 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589363+0000", 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8346e-05 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589381+0000", 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7357e-05 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640373+0000", 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050992347 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640504+0000", 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000131011 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640727+0000", 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022310799999999999 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.605 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64352 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974319594000001, 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051932649999999997, 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64352 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589402+0000", 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.237e-06 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589408+0000", 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2770000000000002e-06 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589426+0000", 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8006e-05 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589441+0000", 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4934e-05 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640803+0000", 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051361619999999997 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640986+0000", 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018303199999999999 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641333+0000", 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034754399999999998 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.609 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64353 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974261200000001, 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052565148999999999, 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64353 2024-09-18T00:58:36.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589460+0000", 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4300000000000001e-06 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589466+0000", 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.446e-06 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589485+0000", 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8726999999999999e-05 2024-09-18T00:58:36.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589499+0000", 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.402e-05 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641427+0000", 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051927945000000003 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641601+0000", 2024-09-18T00:58:36.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174859 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642024+0000", 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000422722 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64354 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974186501, 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053055607999999997, 2024-09-18T00:58:36.614 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64354 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589535+0000", 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3430000000000001e-06 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589542+0000", 2024-09-18T00:58:36.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589561+0000", 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9207000000000001e-05 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589577+0000", 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5933000000000001e-05 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642111+0000", 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052533799999999999 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642271+0000", 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015998499999999999 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642589+0000", 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031818999999999999 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.618 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64355 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974122465000001, 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053570327000000001, 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.619 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64355 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589599+0000", 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589605+0000", 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8830000000000003e-06 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589622+0000", 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.732e-05 2024-09-18T00:58:36.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589636+0000", 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3703000000000001e-05 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642672+0000", 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053036204000000003 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642828+0000", 2024-09-18T00:58:36.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155451 2024-09-18T00:58:36.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643168+0000", 2024-09-18T00:58:36.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034051900000000001 2024-09-18T00:58:36.623 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.623 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.623 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64356 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974064951999999, 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054203299000000003, 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64356 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589656+0000", 2024-09-18T00:58:36.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589662+0000", 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9800000000000003e-06 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589681+0000", 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8300000000000001e-05 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589695+0000", 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3997e-05 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643249+0000", 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053554101999999999 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643423+0000", 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174062 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643859+0000", 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043562499999999998 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64357 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.974007002, 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054239849, 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.628 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64357 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589714+0000", 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.266e-06 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589720+0000", 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2340000000000002e-06 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589738+0000", 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7932999999999999e-05 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589752+0000", 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4867e-05 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643934+0000", 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054181721000000002 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643953+0000", 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8828e-05 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64358 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.973949123000001, 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054261125, 2024-09-18T00:58:36.632 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64358 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589772+0000", 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1170000000000001e-06 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589778+0000", 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8429999999999998e-06 2024-09-18T00:58:36.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589799+0000", 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0727e-05 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589813+0000", 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4403000000000001e-05 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644021+0000", 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054207784000000002 2024-09-18T00:58:36.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644032+0000", 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1251e-05 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64359 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.973887157, 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286586999999997, 2024-09-18T00:58:36.636 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64359 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589834+0000", 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589841+0000", 2024-09-18T00:58:36.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.956e-06 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589864+0000", 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3124000000000001e-05 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589879+0000", 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.486e-05 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644099+0000", 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054219429 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644120+0000", 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1004000000000001e-05 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64360 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.973822963, 2024-09-18T00:58:36.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054290699999999997, 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64360 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589898+0000", 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3600000000000002e-07 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589903+0000", 2024-09-18T00:58:36.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4669999999999998e-06 2024-09-18T00:58:36.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589913+0000", 2024-09-18T00:58:36.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0463e-05 2024-09-18T00:58:36.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589921+0000", 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7959999999999996e-06 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644173+0000", 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054251903999999997 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644188+0000", 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5133999999999999e-05 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.644 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64361 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.973789483999999, 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054320101000000003, 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.645 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64361 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5000000000000002e-07 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589935+0000", 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2799999999999999e-06 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589944+0000", 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589954+0000", 2024-09-18T00:58:36.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0036e-05 2024-09-18T00:58:36.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644240+0000", 2024-09-18T00:58:36.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286273000000003 2024-09-18T00:58:36.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644251+0000", 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0392e-05 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64362 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.973756337999999, 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054340196, 2024-09-18T00:58:36.649 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64362 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589978+0000", 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3570000000000001e-05 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589991+0000", 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3196999999999999e-05 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590000+0000", 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2800000000000003e-06 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644294+0000", 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054294873 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644304+0000", 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6560000000000004e-06 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64363 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.973710307999999, 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054356660000000001, 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:36.653 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64363 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:36.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590011+0000", 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7400000000000003e-07 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590014+0000", 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590024+0000", 2024-09-18T00:58:36.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5670000000000001e-06 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590031+0000", 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3830000000000003e-06 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644354+0000", 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054323429999999999 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644367+0000", 2024-09-18T00:58:36.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.217e-05 2024-09-18T00:58:36.657 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.657 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.657 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.657 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:36.657 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:36.657 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:36.658 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:36.659 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:36.659 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:36.653+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:36.667 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:36.667 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:36.683 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:36.683 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:36.701 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:36.701 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:36.717 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-18T00:58:36.725 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:36.725 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:36.746 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:36.746 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:36.759 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:36.753+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:36.766 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:36.766 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:36.787 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:36.787 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:36.829 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone25 from 0xadeb15a (overlap 0x1c4f41) to 0x0 2024-09-18T00:58:36.837 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:36.837 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:36.855 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:36.905 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:36.901+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:36.961 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:36.953+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:36.972 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:36.972 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:36.987 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:37.077 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:37.073+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:37.103 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-18T00:58:37.144 INFO:teuthology.orchestra.run.smithi042.stdout:3103 punch from 0x65a2b8 to 0x66083b, (0x6583 bytes) 2024-09-18T00:58:37.153 INFO:teuthology.orchestra.run.smithi042.stdout:3104 punch from 0x65d5920 to 0x65db0a7, (0x5787 bytes) 2024-09-18T00:58:37.154 INFO:teuthology.orchestra.run.smithi042.stdout:3105 writesame 0xa3935ab thru 0xa39c928 data_size 0x2a(0x937e bytes) 2024-09-18T00:58:37.159 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:37.159 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:37.162 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:37.162 INFO:teuthology.orchestra.run.smithi042.stdout:3107 read 0x4202ad3 thru 0x4211f14 (0xf442 bytes) 2024-09-18T00:58:37.163 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:37.157+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:37.163 INFO:teuthology.orchestra.run.smithi042.stdout:3108 read 0x34cff90 thru 0x34d6152 (0x61c3 bytes) 2024-09-18T00:58:37.165 INFO:teuthology.orchestra.run.smithi042.stdout:3109 read 0x9f0acb3 thru 0x9f18838 (0xdb86 bytes) 2024-09-18T00:58:37.166 INFO:teuthology.orchestra.run.smithi042.stdout:3112 read 0x777285e thru 0x777c018 (0x97bb bytes) 2024-09-18T00:58:37.168 INFO:teuthology.orchestra.run.smithi042.stdout:3113 write 0x5604eeb thru 0x5610f10 (0xc026 bytes) 2024-09-18T00:58:37.176 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:37.187 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:37.187 INFO:teuthology.orchestra.run.smithi042.stdout:3115 write 0x594de30 thru 0x5954aa3 (0x6c74 bytes) 2024-09-18T00:58:37.197 INFO:teuthology.orchestra.run.smithi042.stdout:3116 writesame 0x72f0bb9 thru 0x72f9397 data_size 0x1(0x87df bytes) 2024-09-18T00:58:37.250 INFO:teuthology.orchestra.run.smithi042.stdout:3117 writesame 0xac1e88e thru 0xac225d5 data_size 0x4(0x3d48 bytes) 2024-09-18T00:58:37.263 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:37.257+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:37.270 INFO:teuthology.orchestra.run.smithi042.stdout:3119 write 0xb12c13f thru 0xb132a03 (0x68c5 bytes) 2024-09-18T00:58:37.277 INFO:teuthology.orchestra.run.smithi042.stdout:3120 read 0x6cd3cd4 thru 0x6cdc304 (0x8631 bytes) 2024-09-18T00:58:37.289 INFO:teuthology.orchestra.run.smithi042.stdout:3121 trunc from 0xb132a04 to 0xe588d1c 2024-09-18T00:58:37.329 INFO:teuthology.orchestra.run.smithi042.stdout:3122 trunc from 0xe588d1c to 0x36f702c 2024-09-18T00:58:37.338 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:37.338 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:37.356 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:37.387 INFO:teuthology.orchestra.run.smithi042.stdout:3123 writesame 0x29e278f thru 0x29eb66a data_size 0x4(0x8edc bytes) 2024-09-18T00:58:37.397 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:37.393+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:37.413 INFO:teuthology.orchestra.run.smithi042.stdout:3124 trunc from 0x36f702c to 0xa0b16ff 2024-09-18T00:58:37.443 INFO:teuthology.orchestra.run.smithi042.stdout:3126 trunc from 0xa0b16ff to 0x2e9d276 2024-09-18T00:58:37.462 INFO:teuthology.orchestra.run.smithi042.stdout:3127 read 0x283debd thru 0x2846413 (0x8557 bytes) 2024-09-18T00:58:37.465 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:37.461+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:37.475 INFO:teuthology.orchestra.run.smithi042.stdout:3128 writesame 0x326e7fc thru 0x327cc17 data_size 0x1a(0xe41c bytes) 2024-09-18T00:58:37.508 INFO:teuthology.orchestra.run.smithi042.stdout:3130 write 0x9f58709 thru 0x9f5cf7b (0x4873 bytes) 2024-09-18T00:58:37.516 INFO:teuthology.orchestra.run.smithi042.stdout:3131 read 0x7c79288 thru 0x7c84173 (0xaeec bytes) 2024-09-18T00:58:37.516 INFO:teuthology.orchestra.run.smithi042.stdout:3133 read 0x9585983 thru 0x9591625 (0xbca3 bytes) 2024-09-18T00:58:37.527 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:37.527 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:37.536 INFO:teuthology.orchestra.run.smithi042.stdout:3134 trunc from 0x9f5cf7c to 0x1f5e41e 2024-09-18T00:58:37.542 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:37.579 INFO:teuthology.orchestra.run.smithi042.stdout:3135 trunc from 0x1f5e41e to 0x6eb2994 2024-09-18T00:58:37.585 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:37.581+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:37.612 INFO:teuthology.orchestra.run.smithi042.stdout:3136 trunc from 0x6eb2994 to 0xde263b6 2024-09-18T00:58:37.623 INFO:teuthology.orchestra.run.smithi042.stdout:3138 write 0x1cbbbeb thru 0x1cc5b94 (0x9faa bytes) 2024-09-18T00:58:37.627 INFO:teuthology.orchestra.run.smithi042.stdout:3139 read 0x9d070fa thru 0x9d14eb9 (0xddc0 bytes) 2024-09-18T00:58:37.627 INFO:teuthology.orchestra.run.smithi042.stdout:3140 punch from 0xcb6e625 to 0xcb71678, (0x3053 bytes) 2024-09-18T00:58:37.640 INFO:teuthology.orchestra.run.smithi042.stdout:3142 trunc from 0xde263b6 to 0x933483b 2024-09-18T00:58:37.661 INFO:teuthology.orchestra.run.smithi042.stdout:3143 trunc from 0x933483b to 0xca4708b 2024-09-18T00:58:37.668 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:37.661+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:37.672 INFO:teuthology.orchestra.run.smithi042.stdout:3144 write 0x97f5b8c thru 0x9800286 (0xa6fb bytes) 2024-09-18T00:58:37.680 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:37.680 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:37.680 INFO:teuthology.orchestra.run.smithi042.stdout:3148 writesame 0x27ffb14 thru 0x2801431 data_size 0x5(0x191e bytes) 2024-09-18T00:58:37.684 INFO:teuthology.orchestra.run.smithi042.stdout:3149 punch from 0xa1427a4 to 0xa144208, (0x1a64 bytes) 2024-09-18T00:58:37.684 INFO:teuthology.orchestra.run.smithi042.stdout:3150 read 0xc3cc33f thru 0xc3d8488 (0xc14a bytes) 2024-09-18T00:58:37.684 INFO:teuthology.orchestra.run.smithi042.stdout:3152 punch from 0x151e3c8 to 0x1525228, (0x6e60 bytes) 2024-09-18T00:58:37.684 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:37.684 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:37.696 INFO:teuthology.orchestra.run.smithi042.stdout:3156 write 0xd493bbc thru 0xd49bfb5 (0x83fa bytes) 2024-09-18T00:58:37.702 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:37.702 INFO:teuthology.orchestra.run.smithi042.stdout:3159 writesame 0x6037889 thru 0x60437cf data_size 0x1(0xbf47 bytes) 2024-09-18T00:58:37.710 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:37.710 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:37.726 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:37.729 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:58:37.751 INFO:teuthology.orchestra.run.smithi042.stdout:3162 writesame 0x52dd07 thru 0x53083b data_size 0x3(0x2b35 bytes) 2024-09-18T00:58:37.758 INFO:teuthology.orchestra.run.smithi042.stdout:3163 writesame 0x6e2b1dc thru 0x6e3770a data_size 0xd(0xc52f bytes) 2024-09-18T00:58:37.768 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:37.761+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:37.772 INFO:teuthology.orchestra.run.smithi042.stdout:3164 write 0xddb0dd7 thru 0xddb4707 (0x3931 bytes) 2024-09-18T00:58:37.777 INFO:teuthology.orchestra.run.smithi042.stdout:3165 clone 27 order 25 su 8388608 sc 8 2024-09-18T00:58:37.863 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:37.863 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:37.869 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:37.861+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:37.877 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:37.970 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:37.965+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:38.042 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:38.042 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:38.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:38.070 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:38.069+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:38.171 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:38.165+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:38.183 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:38.183 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:38.198 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:38.272 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:38.265+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:38.326 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:38.326 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:38.340 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:38.386 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:38.381+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:38.468 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:38.468 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:38.474 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:38.469+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:38.482 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:38.575 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:38.573+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:38.654 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:38.654 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:38.668 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:38.714 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:38.709+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:38.725 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:58:38.776 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:38.769+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:38.805 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:38.805 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:38.807 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:38.822 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:38.877 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:38.873+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:38.908 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone26 from 0xddb4708 (overlap 0x1f5e41e) to 0x0 2024-09-18T00:58:38.960 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:38.977 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:38.977 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:38.978 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:38.973+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:38.993 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:38.993 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:38.998 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:38.998 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:39.010 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:39.015 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:39.015 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:39.041 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:39.041 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:39.061 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-18T00:58:39.062 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:39.063 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:39.073 INFO:teuthology.orchestra.run.smithi042.stdout:3168 flatten 2024-09-18T00:58:39.078 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:39.073+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:39.091 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:39.118 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:39.118 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:39.140 INFO:teuthology.orchestra.run.smithi042.stdout:3169 trunc from 0xddb4708 to 0x15e8fe7 2024-09-18T00:58:39.145 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:39.145 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:39.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.066676004999998, 2024-09-18T00:58:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:58:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.065746414000003, 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:58:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:58:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:58:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:58:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:58:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:58:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:58:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:58:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.065732353999998, 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:58:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:58:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:58:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:58:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:58:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:58:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:58:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:58:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.064441778999999, 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:58:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.064421307000003, 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:58:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:58:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.061626853999996, 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.061611669999998, 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:58:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:58:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.061588724000003, 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.057411084999998, 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:58:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:58:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.057389868000001, 2024-09-18T00:58:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:58:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.057368484999998, 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:58:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:58:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:58:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:58:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:58:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:58:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:58:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.053468631000001, 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.053454430000002, 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:58:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:58:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:58:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:58:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.053436392999998, 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.053422327, 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:58:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:58:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.049396975999997, 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.049382964000003, 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:58:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.049368999999999, 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.287511045999999, 2024-09-18T00:58:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:58:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:58:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.162034675000001, 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:58:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:58:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.233 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:39.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:39.235 INFO:teuthology.orchestra.run.smithi042.stdout:3170 write 0x95c3bb6 thru 0x95cd59c (0x99e7 bytes) 2024-09-18T00:58:39.235 INFO:teuthology.orchestra.run.smithi042.stdout:3171 trunc from 0x95cd59d to 0xca3636c 2024-09-18T00:58:39.235 INFO:teuthology.orchestra.run.smithi042.stdout:3173 read 0xb486c9d thru 0xb491985 (0xace9 bytes) 2024-09-18T00:58:39.235 INFO:teuthology.orchestra.run.smithi042.stdout:3174 read 0x3344689 thru 0x3350867 (0xc1df bytes) 2024-09-18T00:58:39.235 INFO:teuthology.orchestra.run.smithi042.stdout:3175 writesame 0x93929b4 thru 0x939b802 data_size 0x1(0x8e4f bytes) 2024-09-18T00:58:39.236 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:39.229+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:39.261 INFO:teuthology.orchestra.run.smithi042.stdout:3176 read 0x9b35132 thru 0x9b373b0 (0x227f bytes) 2024-09-18T00:58:39.261 INFO:teuthology.orchestra.run.smithi042.stdout:3177 writesame 0x7cd5813 thru 0x7ce051d data_size 0x1(0xad0b bytes) 2024-09-18T00:58:39.315 INFO:teuthology.orchestra.run.smithi042.stdout:3178 read 0x2c22230 thru 0x2c2a1d1 (0x7fa2 bytes) 2024-09-18T00:58:39.315 INFO:teuthology.orchestra.run.smithi042.stdout:3179 write 0xb607de8 thru 0xb615b11 (0xdd2a bytes) 2024-09-18T00:58:39.323 INFO:teuthology.orchestra.run.smithi042.stdout:3180 write 0x95e1281 thru 0x95ecb73 (0xb8f3 bytes) 2024-09-18T00:58:39.327 INFO:teuthology.orchestra.run.smithi042.stdout:3181 read 0x459f571 thru 0x45a4feb (0x5a7b bytes) 2024-09-18T00:58:39.328 INFO:teuthology.orchestra.run.smithi042.stdout:3182 punch from 0x5584edd to 0x5586efe, (0x2021 bytes) 2024-09-18T00:58:39.335 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:39.329+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:39.351 INFO:teuthology.orchestra.run.smithi042.stdout:3183 trunc from 0xca3636c to 0x25256d7 2024-09-18T00:58:39.381 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:73129 3.17 e71/24 3:ee14d8ee:::rbd_object_map.117c304cc916:head v 71'288, mlcod=71'286)", 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:39.394608+0000", 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.00053997099999999998, 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00056079000000000003, 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:39.394608+0000", 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:39.394608+0000", 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:39.394611+0000", 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0740000000000001e-06 2024-09-18T00:58:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:39.394635+0000", 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3512000000000001e-05 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:39.394672+0000", 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6789999999999998e-05 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:39.394690+0000", 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8196999999999999e-05 2024-09-18T00:58:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:39.394844+0000", 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015424500000000001 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:39.394885+0000", 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0880000000000002e-05 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:58:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 1 2024-09-18T00:58:39.400 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:39.407 INFO:teuthology.orchestra.run.smithi042.stdout:3185 write 0x7d1bf9c thru 0x7d2a8bf (0xe924 bytes) 2024-09-18T00:58:39.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:39.414 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.415 INFO:teuthology.orchestra.run.smithi042.stdout:3188 writesame 0x5a7a6fc thru 0x5a856d9 data_size 0x1(0xafde bytes) 2024-09-18T00:58:39.436 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:39.429+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:39.476 INFO:teuthology.orchestra.run.smithi042.stdout:3190 writesame 0x74b55e1 thru 0x74b5b7c data_size 0x2(0x59c bytes) 2024-09-18T00:58:39.483 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.483 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.483 INFO:teuthology.orchestra.run.smithi042.stdout:3196 read 0x4c3b9a5 thru 0x4c3ef9a (0x35f6 bytes) 2024-09-18T00:58:39.484 INFO:teuthology.orchestra.run.smithi042.stdout:3197 read 0x76dc9d9 thru 0x76dcd87 (0x3af bytes) 2024-09-18T00:58:39.509 INFO:teuthology.orchestra.run.smithi042.stdout:3198 write 0xea1f5bb thru 0xea26fc9 (0x7a0f bytes) 2024-09-18T00:58:39.515 INFO:teuthology.orchestra.run.smithi042.stdout:3200 punch from 0x6adf2d7 to 0x6ae8f8b, (0x9cb4 bytes) 2024-09-18T00:58:39.515 INFO:teuthology.orchestra.run.smithi042.stdout:3202 punch from 0xb77ecd2 to 0xb78a7af, (0xbadd bytes) 2024-09-18T00:58:39.516 INFO:teuthology.orchestra.run.smithi042.stdout:3204 writesame 0x7550a0 thru 0x75f1ec data_size 0x11(0xa14d bytes) 2024-09-18T00:58:39.520 INFO:teuthology.orchestra.run.smithi042.stdout:3205 read 0x819397c thru 0x81a303b (0xf6c0 bytes) 2024-09-18T00:58:39.521 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.521 INFO:teuthology.orchestra.run.smithi042.stdout:3209 write 0x8a26e83 thru 0x8a2f232 (0x83b0 bytes) 2024-09-18T00:58:39.523 INFO:teuthology.orchestra.run.smithi042.stdout:3210 read 0xc3d80d2 thru 0xc3de940 (0x686f bytes) 2024-09-18T00:58:39.524 INFO:teuthology.orchestra.run.smithi042.stdout:3212 punch from 0x81020a to 0x81810c, (0x7f02 bytes) 2024-09-18T00:58:39.524 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.524 INFO:teuthology.orchestra.run.smithi042.stdout:3214 writesame 0xbfc49d thru 0xbfefca data_size 0x1(0x2b2e bytes) 2024-09-18T00:58:39.537 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:39.529+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:39.545 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:39.546 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:39.558 INFO:teuthology.orchestra.run.smithi042.stdout:3215 trunc from 0xea26fca to 0x2ec96b9 2024-09-18T00:58:39.567 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:39.584 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.624 INFO:teuthology.orchestra.run.smithi042.stdout:3218 write 0xe7a0271 thru 0xe7ad41c (0xd1ac bytes) 2024-09-18T00:58:39.637 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:39.629+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:39.638 INFO:teuthology.orchestra.run.smithi042.stdout:3219 trunc from 0xe7ad41d to 0xb4e42f9 2024-09-18T00:58:39.654 INFO:teuthology.orchestra.run.smithi042.stdout:3220 writesame 0x180eac0 thru 0x18163bb data_size 0x59(0x78fc bytes) 2024-09-18T00:58:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:3222 punch from 0x99f92d6 to 0x9a0822a, (0xef54 bytes) 2024-09-18T00:58:39.657 INFO:teuthology.orchestra.run.smithi042.stdout:3223 read 0xa80c9de thru 0xa81253b (0x5b5e bytes) 2024-09-18T00:58:39.658 INFO:teuthology.orchestra.run.smithi042.stdout:3224 read 0x249c347 thru 0x249c639 (0x2f3 bytes) 2024-09-18T00:58:39.659 INFO:teuthology.orchestra.run.smithi042.stdout:3226 write 0x7b0d10 thru 0x7b1b8f (0xe80 bytes) 2024-09-18T00:58:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:3236 punch from 0xdfd46a to 0xdfe307, (0xe9d bytes) 2024-09-18T00:58:39.662 INFO:teuthology.orchestra.run.smithi042.stdout:3237 write 0x120335b thru 0x121302f (0xfcd5 bytes) 2024-09-18T00:58:39.680 INFO:teuthology.orchestra.run.smithi042.stdout:3238 write 0xc1380cb thru 0xc13b6bb (0x35f1 bytes) 2024-09-18T00:58:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:3241 read 0x31bb4e0 thru 0x31c9a52 (0xe573 bytes) 2024-09-18T00:58:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:3242 read 0x67a8ba0 thru 0x67b576e (0xcbcf bytes) 2024-09-18T00:58:39.685 INFO:teuthology.orchestra.run.smithi042.stdout:3243 read 0x1c17d50 thru 0x1c27c8a (0xff3b bytes) 2024-09-18T00:58:39.686 INFO:teuthology.orchestra.run.smithi042.stdout:3245 read 0x681e1ee thru 0x681e7d3 (0x5e6 bytes) 2024-09-18T00:58:39.687 INFO:teuthology.orchestra.run.smithi042.stdout:3247 read 0x2ca8 thru 0x7d93 (0x50ec bytes) 2024-09-18T00:58:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:3250 write 0x320f135 thru 0x32181e3 (0x90af bytes) 2024-09-18T00:58:39.695 INFO:teuthology.orchestra.run.smithi042.stdout:3251 read 0x765e15f thru 0x766daf6 (0xf998 bytes) 2024-09-18T00:58:39.697 INFO:teuthology.orchestra.run.smithi042.stdout:3252 write 0x41209e8 thru 0x412ffa1 (0xf5ba bytes) 2024-09-18T00:58:39.700 INFO:teuthology.orchestra.run.smithi042.stdout:3253 punch from 0x453cc69 to 0x453eb19, (0x1eb0 bytes) 2024-09-18T00:58:39.713 INFO:teuthology.orchestra.run.smithi042.stdout:3254 trunc from 0xc13b6bc to 0x64994b7 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 72.397546903999995, 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101264911, 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4226", 2024-09-18T00:58:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2155402750", 2024-09-18T00:58:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313760+0000", 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.612e-06 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313768+0000", 2024-09-18T00:58:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9249999999999995e-06 2024-09-18T00:58:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313786+0000", 2024-09-18T00:58:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8253999999999998e-05 2024-09-18T00:58:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313803+0000", 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313935+0000", 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013180399999999999 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313954+0000", 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8768999999999999e-05 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.411874+0000", 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.097920646 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412292+0000", 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041777699999999998 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412675+0000", 2024-09-18T00:58:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038262300000000001 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414019+0000", 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013436699999999999 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414872+0000", 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085320999999999997 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414955+0000", 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3405999999999994e-05 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.415023+0000", 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7825999999999997e-05 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15500 3.11 3:8ed6a97b:::rbd_data.117c53e529c1.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.896786884000001, 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046818956000000002, 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15500 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814521+0000", 2024-09-18T00:58:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.977e-06 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814536+0000", 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.487e-05 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814572+0000", 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5567e-05 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814588+0000", 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6546e-05 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814675+0000", 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6675999999999995e-05 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814839+0000", 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016414199999999999 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.819908+0000", 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050695280000000002 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861139+0000", 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041230771999999999 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861208+0000", 2024-09-18T00:58:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8722999999999999e-05 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861277+0000", 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9246999999999996e-05 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861337+0000", 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9908000000000003e-05 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26860 3.a 3:523ff797:::rbd_data.117c4c7db1a0.0000000000000039:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.338612765000001, 2024-09-18T00:58:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028475341000000001, 2024-09-18T00:58:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26860 2024-09-18T00:58:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372693+0000", 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0550000000000001e-06 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372699+0000", 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372714+0000", 2024-09-18T00:58:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4673e-05 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372722+0000", 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4659999999999999e-06 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372787+0000", 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5288000000000002e-05 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372874+0000", 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6485999999999998e-05 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.377992+0000", 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051177159999999996 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401008+0000", 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02301603 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401026+0000", 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712e-05 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401082+0000", 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5648e-05 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401168+0000", 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5476999999999995e-05 2024-09-18T00:58:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64289 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 3227648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.124923069999999, 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025443502999999999, 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64289 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.84e-07 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586385+0000", 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2060000000000001e-06 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586392+0000", 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586396+0000", 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5859999999999998e-06 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611580+0000", 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025183646000000001 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611667+0000", 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7350999999999998e-05 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611825+0000", 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015804999999999999 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64403 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 475136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123976587, 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025296714000000001, 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64403 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587329+0000", 2024-09-18T00:58:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587331+0000", 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9889999999999999e-06 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587338+0000", 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5209999999999997e-06 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587343+0000", 2024-09-18T00:58:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612401+0000", 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025057505000000001 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612478+0000", 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6817999999999995e-05 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612625+0000", 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014740099999999999 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64404 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 655360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.1239553, 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025540833999999998, 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64404 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587352+0000", 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587362+0000", 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0125e-05 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587367+0000", 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1150000000000002e-06 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612671+0000", 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025303638 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612751+0000", 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0107999999999999e-05 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612891+0000", 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013946 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64405 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123929945, 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025814888000000001, 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64405 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587379+0000", 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1439999999999999e-06 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587398+0000", 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587405+0000", 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3740000000000003e-06 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612931+0000", 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025526604000000001 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613008+0000", 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6508000000000007e-05 2024-09-18T00:58:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613190+0000", 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018205200000000001 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64406 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123877945, 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026015033, 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64406 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587428+0000", 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0399999999999996e-07 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587431+0000", 2024-09-18T00:58:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1870000000000001e-06 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587441+0000", 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0349999999999999e-05 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587447+0000", 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.083e-06 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613236+0000", 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025788953 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613288+0000", 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2275999999999999e-05 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613442+0000", 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015358 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64407 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123848946000001, 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026326130999999999, 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64407 2024-09-18T00:58:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587459+0000", 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3750000000000001e-06 2024-09-18T00:58:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587465+0000", 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7150000000000003e-06 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587469+0000", 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8060000000000002e-06 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613488+0000", 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026018645999999999 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613576+0000", 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7892000000000005e-05 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613782+0000", 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000206233 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64408 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123828702999999, 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026568468000000001, 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64408 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587477+0000", 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7500000000000001e-07 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587479+0000", 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1900000000000002e-06 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587487+0000", 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1869999999999997e-06 2024-09-18T00:58:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587492+0000", 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3149999999999999e-06 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613823+0000", 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026330339000000001 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613885+0000", 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2391999999999995e-05 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614045+0000", 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015966999999999999 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64409 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123805726000001, 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026785499000000001, 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64409 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587500+0000", 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587502+0000", 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9420000000000002e-06 2024-09-18T00:58:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587508+0000", 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1840000000000004e-06 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587513+0000", 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9640000000000002e-06 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614075+0000", 2024-09-18T00:58:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026561889000000002 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614124+0000", 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9177e-05 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614285+0000", 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016103100000000001 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64410 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123785777, 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026987559000000001, 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64410 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587520+0000", 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587522+0000", 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1500000000000002e-06 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587529+0000", 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5839999999999997e-06 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587534+0000", 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.826e-06 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614317+0000", 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026783065000000002 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614385+0000", 2024-09-18T00:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7682999999999996e-05 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614507+0000", 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121859 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64411 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123764292000001, 2024-09-18T00:58:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027224245000000001, 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64411 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587543+0000", 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587550+0000", 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5810000000000002e-06 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587555+0000", 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0490000000000003e-06 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614543+0000", 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026988056 2024-09-18T00:58:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614619+0000", 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5932000000000004e-05 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614765+0000", 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014603899999999999 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64412 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2097152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123743547, 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027478605, 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64412 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587562+0000", 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6499999999999999e-07 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587564+0000", 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3300000000000001e-06 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587571+0000", 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2949999999999999e-06 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587575+0000", 2024-09-18T00:58:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6360000000000002e-06 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614798+0000", 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027222860000000002 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614874+0000", 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5383000000000007e-05 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615040+0000", 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016653600000000001 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64413 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123722743, 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027709421000000001, 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64413 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587583+0000", 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587585+0000", 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-18T00:58:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587592+0000", 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9700000000000002e-06 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587597+0000", 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8289999999999998e-06 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615071+0000", 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027474218000000002 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615143+0000", 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1505000000000001e-05 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615292+0000", 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014900199999999999 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64414 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2457600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123701205, 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027920637000000002, 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64414 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.08e-07 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587606+0000", 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1299999999999999e-06 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587612+0000", 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1319999999999999e-06 2024-09-18T00:58:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587618+0000", 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.147e-06 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615323+0000", 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027705423 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615366+0000", 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2914000000000003e-05 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615524+0000", 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015848300000000001 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64415 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123676884, 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028241265000000002, 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64415 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587631+0000", 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5550000000000001e-06 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587639+0000", 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.807e-06 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587644+0000", 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2630000000000003e-06 2024-09-18T00:58:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615633+0000", 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027989342 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615703+0000", 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9956000000000006e-05 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615869+0000", 2024-09-18T00:58:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016597500000000001 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64416 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2818048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123653587, 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028575605, 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64416 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587652+0000", 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587655+0000", 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587662+0000", 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7789999999999996e-06 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587668+0000", 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2109999999999997e-06 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615916+0000", 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028248412000000001 2024-09-18T00:58:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616009+0000", 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2682999999999993e-05 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616227+0000", 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218345 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64417 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123629576000001, 2024-09-18T00:58:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028891495, 2024-09-18T00:58:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64417 2024-09-18T00:58:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587676+0000", 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.34e-07 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587683+0000", 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8739999999999997e-06 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587692+0000", 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2159999999999995e-06 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587698+0000", 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.716e-06 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616259+0000", 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028561733999999998 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616369+0000", 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110104 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616567+0000", 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000197417 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64418 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.123599132000001, 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029401729000000001, 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64418 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8400000000000004e-07 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587709+0000", 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5390000000000002e-06 2024-09-18T00:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587717+0000", 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0819999999999999e-06 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587722+0000", 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4140000000000002e-06 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616869+0000", 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029146763999999999 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616926+0000", 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7250000000000002e-05 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.617108+0000", 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181096 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:39.812 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:39.813 INFO:teuthology.orchestra.run.smithi042.stdout:3258 writesame 0x28289e7 thru 0x28323a4 data_size 0xb(0x99be bytes) 2024-09-18T00:58:39.814 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:39.815 INFO:teuthology.orchestra.run.smithi042.stdout:3259 punch from 0x1ed1c00 to 0x1ed4de5, (0x31e5 bytes) 2024-09-18T00:58:39.816 INFO:teuthology.orchestra.run.smithi042.stdout:3260 write 0x84e61f6 thru 0x84ebef7 (0x5d02 bytes) 2024-09-18T00:58:39.816 INFO:teuthology.orchestra.run.smithi042.stdout:3261 write 0xae92465 thru 0xae98533 (0x60cf bytes) 2024-09-18T00:58:39.816 INFO:teuthology.orchestra.run.smithi042.stdout:3262 write 0xc0e5e3 thru 0xc154bf (0x6edd bytes) 2024-09-18T00:58:39.816 INFO:teuthology.orchestra.run.smithi042.stdout:3263 read 0x19035d8 thru 0x19070bb (0x3ae4 bytes) 2024-09-18T00:58:39.816 INFO:teuthology.orchestra.run.smithi042.stdout:3264 read 0x69725a9 thru 0x697b003 (0x8a5b bytes) 2024-09-18T00:58:39.816 INFO:teuthology.orchestra.run.smithi042.stdout:3265 punch from 0x5b53aa1 to 0x5b5599c, (0x1efb bytes) 2024-09-18T00:58:39.816 INFO:teuthology.orchestra.run.smithi042.stdout:3267 write 0x5fba192 thru 0x5fbd793 (0x3602 bytes) 2024-09-18T00:58:39.816 INFO:teuthology.orchestra.run.smithi042.stdout:3268 read 0x3f49ace thru 0x3f4eea3 (0x53d6 bytes) 2024-09-18T00:58:39.816 INFO:teuthology.orchestra.run.smithi042.stdout:3269 writesame 0xa19073 thru 0xa23834 data_size 0x2(0xa7c2 bytes) 2024-09-18T00:58:39.816 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:39.809+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:39.828 INFO:teuthology.orchestra.run.smithi042.stdout:3271 write 0xdd477b3 thru 0xdd52dd5 (0xb623 bytes) 2024-09-18T00:58:39.832 INFO:teuthology.orchestra.run.smithi042.stdout:3272 write 0x8a82851 thru 0x8a8a284 (0x7a34 bytes) 2024-09-18T00:58:39.836 INFO:teuthology.orchestra.run.smithi042.stdout:3274 read 0x36106e4 thru 0x361eca6 (0xe5c3 bytes) 2024-09-18T00:58:39.837 INFO:teuthology.orchestra.run.smithi042.stdout:3275 writesame 0x3bdc9b4 thru 0x3be510b data_size 0x8(0x8758 bytes) 2024-09-18T00:58:39.849 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.849 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.849 INFO:teuthology.orchestra.run.smithi042.stdout:3280 write 0xbe1fc61 thru 0xbe23e1b (0x41bb bytes) 2024-09-18T00:58:39.851 INFO:teuthology.orchestra.run.smithi042.stdout:3282 read 0x89186f5 thru 0x891c33c (0x3c48 bytes) 2024-09-18T00:58:39.852 INFO:teuthology.orchestra.run.smithi042.stdout:3283 read 0x28cb5bf thru 0x28cd143 (0x1b85 bytes) 2024-09-18T00:58:39.866 INFO:teuthology.orchestra.run.smithi042.stdout:3286 trunc from 0xdd52dd6 to 0x9166cff 2024-09-18T00:58:39.883 INFO:teuthology.orchestra.run.smithi042.stdout:3289 writesame 0x51663a0 thru 0x517194a data_size 0x1(0xb5ab bytes) 2024-09-18T00:58:39.904 INFO:teuthology.orchestra.run.smithi042.stdout:3291 write 0x69273b0 thru 0x6935335 (0xdf86 bytes) 2024-09-18T00:58:39.908 INFO:teuthology.orchestra.run.smithi042.stdout:3292 read 0x8ee6f8b thru 0x8eeffda (0x9050 bytes) 2024-09-18T00:58:39.909 INFO:teuthology.orchestra.run.smithi042.stdout:3293 read 0x32a0b83 thru 0x32a656a (0x59e8 bytes) 2024-09-18T00:58:39.910 INFO:teuthology.orchestra.run.smithi042.stdout:3294 punch from 0x22b233f to 0x22b2b60, (0x821 bytes) 2024-09-18T00:58:39.913 INFO:teuthology.orchestra.run.smithi042.stdout:3296 trunc from 0x9166cff to 0x9dff82b 2024-09-18T00:58:39.915 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:39.909+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:3297 punch from 0x1aeeac9 to 0x1af9ba8, (0xb0df bytes) 2024-09-18T00:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.926 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.934 INFO:teuthology.orchestra.run.smithi042.stdout:3301 trunc from 0x9dff82b to 0x72db219 2024-09-18T00:58:39.951 INFO:teuthology.orchestra.run.smithi042.stdout:3302 punch from 0x24c92ae to 0x24d46fd, (0xb44f bytes) 2024-09-18T00:58:39.951 INFO:teuthology.orchestra.run.smithi042.stdout:3303 read 0xd96274 thru 0xd9f683 (0x9410 bytes) 2024-09-18T00:58:39.955 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:39.955 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:39.955 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:39.955 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:39.969 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:39.981 INFO:teuthology.orchestra.run.smithi042.stdout:3304 write 0xcce5a31 thru 0xcce9407 (0x39d7 bytes) 2024-09-18T00:58:39.985 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:39.985 INFO:teuthology.orchestra.run.smithi042.stdout:3307 read 0xbdf9c0f thru 0xbe06d1b (0xd10d bytes) 2024-09-18T00:58:40.002 INFO:teuthology.orchestra.run.smithi042.stdout:3308 write 0xe3a23d7 thru 0xe3a38cd (0x14f7 bytes) 2024-09-18T00:58:40.011 INFO:teuthology.orchestra.run.smithi042.stdout:3309 trunc from 0xe3a38ce to 0xbcc6334 2024-09-18T00:58:40.027 INFO:teuthology.orchestra.run.smithi042.stdout:3310 read 0x62427a7 thru 0x6251c1a (0xf474 bytes) 2024-09-18T00:58:40.027 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.028 INFO:teuthology.orchestra.run.smithi042.stdout:3312 write 0x53f7469 thru 0x53fc1a3 (0x4d3b bytes) 2024-09-18T00:58:40.030 INFO:teuthology.orchestra.run.smithi042.stdout:3313 read 0x3ec2e6b thru 0x3ed141a (0xe5b0 bytes) 2024-09-18T00:58:40.031 INFO:teuthology.orchestra.run.smithi042.stdout:3314 read 0xb4ac8d0 thru 0xb4aec8d (0x23be bytes) 2024-09-18T00:58:40.031 INFO:teuthology.orchestra.run.smithi042.stdout:3315 punch from 0x5b65ee3 to 0x5b67928, (0x1a45 bytes) 2024-09-18T00:58:40.032 INFO:teuthology.orchestra.run.smithi042.stdout:3317 write 0x88902d9 thru 0x88965d7 (0x62ff bytes) 2024-09-18T00:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout:3319 write 0x3ccb24 thru 0x3d1002 (0x44df bytes) 2024-09-18T00:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout:3320 write 0x63160bf thru 0x6324db2 (0xecf4 bytes) 2024-09-18T00:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout:3321 read 0x441805a thru 0x4419f1d (0x1ec4 bytes) 2024-09-18T00:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout:3322 read 0x29bbe3b thru 0x29be626 (0x27ec bytes) 2024-09-18T00:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout:3324 punch from 0x7d6c4e5 to 0x7d72977, (0x6492 bytes) 2024-09-18T00:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout:3326 writesame 0xd99a12 thru 0xda456d data_size 0x16(0xab5c bytes) 2024-09-18T00:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout:3327 write 0x8f1e0c6 thru 0x8f2bd5a (0xdc95 bytes) 2024-09-18T00:58:40.054 INFO:teuthology.orchestra.run.smithi042.stdout:3328 write 0x14ad933 thru 0x14baf7f (0xd64d bytes) 2024-09-18T00:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:3329 read 0xa0f8fb2 thru 0xa0f9d3b (0xd8a bytes) 2024-09-18T00:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:3331 read 0x552fdfa thru 0x5537aeb (0x7cf2 bytes) 2024-09-18T00:58:40.058 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:40.053+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:3333 write 0x55d9ccb thru 0x55e4d67 (0xb09d bytes) 2024-09-18T00:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout:3334 write 0x87e034f thru 0x87e5ef7 (0x5ba9 bytes) 2024-09-18T00:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:3335 read 0x591d612 thru 0x592d3b2 (0xfda1 bytes) 2024-09-18T00:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout:3336 trunc from 0xbcc6334 to 0x14136a0 2024-09-18T00:58:40.116 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:40.109+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:40.143 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:40.143 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:40.157 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:40.159 INFO:teuthology.orchestra.run.smithi042.stdout:3337 trunc from 0x14136a0 to 0xbb974fd 2024-09-18T00:58:40.173 INFO:teuthology.orchestra.run.smithi042.stdout:3338 write 0x9d01e22 thru 0x9d04fc2 (0x31a1 bytes) 2024-09-18T00:58:40.179 INFO:teuthology.orchestra.run.smithi042.stdout:3339 read 0xa24635c thru 0xa24d60e (0x72b3 bytes) 2024-09-18T00:58:40.199 INFO:teuthology.orchestra.run.smithi042.stdout:3340 trunc from 0xbb974fd to 0x39ce8fa 2024-09-18T00:58:40.214 INFO:teuthology.orchestra.run.smithi042.stdout:3341 read 0xbefd8b thru 0xbfa7aa (0xaa20 bytes) 2024-09-18T00:58:40.215 INFO:teuthology.orchestra.run.smithi042.stdout:3342 punch from 0xa200cf to 0xa229ea, (0x291b bytes) 2024-09-18T00:58:40.215 INFO:teuthology.orchestra.run.smithi042.stdout:3343 write 0x34e71d5 thru 0x34eb857 (0x4683 bytes) 2024-09-18T00:58:40.216 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:40.209+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:40.223 INFO:teuthology.orchestra.run.smithi042.stdout:3346 trunc from 0x39ce8fa to 0x455f797 2024-09-18T00:58:40.234 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.253 INFO:teuthology.orchestra.run.smithi042.stdout:3348 write 0x71f5e4f thru 0x71f61b1 (0x363 bytes) 2024-09-18T00:58:40.256 INFO:teuthology.orchestra.run.smithi042.stdout:3349 read 0x60ddb65 thru 0x60eaa8d (0xcf29 bytes) 2024-09-18T00:58:40.256 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.256 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.256 INFO:teuthology.orchestra.run.smithi042.stdout:3354 read 0x1e83a3 thru 0x1ec358 (0x3fb6 bytes) 2024-09-18T00:58:40.257 INFO:teuthology.orchestra.run.smithi042.stdout:3355 read 0x665b575 thru 0x6667dea (0xc876 bytes) 2024-09-18T00:58:40.257 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.257 INFO:teuthology.orchestra.run.smithi042.stdout:3357 read 0x3bff766 thru 0x3c08a07 (0x92a2 bytes) 2024-09-18T00:58:40.272 INFO:teuthology.orchestra.run.smithi042.stdout:3359 trunc from 0x71f61b2 to 0xcb2f07f 2024-09-18T00:58:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:3360 write 0x27356db thru 0x27376c8 (0x1fee bytes) 2024-09-18T00:58:40.291 INFO:teuthology.orchestra.run.smithi042.stdout:3361 read 0x622b177 thru 0x62310f1 (0x5f7b bytes) 2024-09-18T00:58:40.292 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.292 INFO:teuthology.orchestra.run.smithi042.stdout:3365 write 0x35fd9a0 thru 0x3604c2e (0x728f bytes) 2024-09-18T00:58:40.295 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.295 INFO:teuthology.orchestra.run.smithi042.stdout:3368 read 0x2138443 thru 0x214471c (0xc2da bytes) 2024-09-18T00:58:40.296 INFO:teuthology.orchestra.run.smithi042.stdout:3371 read 0xc308a9b thru 0xc314783 (0xbce9 bytes) 2024-09-18T00:58:40.297 INFO:teuthology.orchestra.run.smithi042.stdout:3373 read 0x643ee85 thru 0x6441182 (0x22fe bytes) 2024-09-18T00:58:40.297 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.298 INFO:teuthology.orchestra.run.smithi042.stdout:3375 writesame 0x56f5c9c thru 0x56ff9d8 data_size 0x1(0x9d3d bytes) 2024-09-18T00:58:40.317 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:40.313+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:40.330 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.332 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:40.332 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:40.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1670 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1335296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.162731629, 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083399740999999999, 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.333 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1670 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166573+0000", 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1829999999999999e-06 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166578+0000", 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0030000000000002e-06 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166594+0000", 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6470999999999999e-05 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166607+0000", 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2831e-05 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249706+0000", 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083099022999999994 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249843+0000", 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136518 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249971+0000", 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012871200000000001 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1671 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1376256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.162680852000001, 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083700574, 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.337 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1671 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166623+0000", 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.066e-06 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166641+0000", 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7728e-05 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166659+0000", 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7850999999999999e-05 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166672+0000", 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3020000000000001e-05 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250034+0000", 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083362085000000002 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250145+0000", 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111045 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250323+0000", 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017777900000000001 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1672 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:40.341 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.162607393000002, 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083853480999999994, 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1672 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:40.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166697+0000", 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0920000000000001e-06 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166702+0000", 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0799999999999996e-06 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166718+0000", 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6167999999999998e-05 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166745+0000", 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7177000000000001e-05 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250382+0000", 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083636792000000001 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250476+0000", 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250549+0000", 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3282999999999996e-05 2024-09-18T00:58:40.345 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1673 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.162540284999999, 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084151880999999998, 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.346 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1673 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166764+0000", 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166770+0000", 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5969999999999999e-06 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166786+0000", 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6439999999999998e-05 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166799+0000", 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2934e-05 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250605+0000", 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083806357999999997 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250724+0000", 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011821500000000001 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250915+0000", 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000191251 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1674 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:40.350 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.162440699999998, 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084239101999999996, 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1674 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166864+0000", 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3629999999999999e-06 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166883+0000", 2024-09-18T00:58:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9083e-05 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166902+0000", 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8711000000000001e-05 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166915+0000", 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3375e-05 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250962+0000", 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084047254000000002 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251029+0000", 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6315000000000005e-05 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251102+0000", 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3001000000000001e-05 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1675 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1540096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.162368849000003, 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084536891000000003, 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1675 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166936+0000", 2024-09-18T00:58:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2720000000000001e-06 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166941+0000", 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7370000000000003e-06 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166958+0000", 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166972+0000", 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3569e-05 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251159+0000", 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084187467000000002 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251300+0000", 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014059200000000001 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251471+0000", 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017168000000000001 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1676 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.162315968999998, 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084738637000000006, 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1676 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166988+0000", 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166993+0000", 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8339999999999998e-06 2024-09-18T00:58:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167009+0000", 2024-09-18T00:58:40.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6028e-05 2024-09-18T00:58:40.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167022+0000", 2024-09-18T00:58:40.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2675e-05 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251517+0000", 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084494975999999999 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251576+0000", 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8644999999999998e-05 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251726+0000", 2024-09-18T00:58:40.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015039 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1677 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.162264297999997, 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085055403000000002, 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1677 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167040+0000", 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167046+0000", 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167062+0000", 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6354e-05 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167075+0000", 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2965e-05 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251789+0000", 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084713940000000001 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251916+0000", 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000127176 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252094+0000", 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017833599999999999 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1678 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.162212535000002, 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085316746999999998, 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.368 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1678 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:40.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167092+0000", 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167097+0000", 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7029999999999997e-06 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167113+0000", 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-18T00:58:40.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167125+0000", 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2517000000000001e-05 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252157+0000", 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085031933000000004 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252277+0000", 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119708 2024-09-18T00:58:40.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252407+0000", 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013031800000000001 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1679 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.162161816000001, 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085557195000000003, 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.372 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1679 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167143+0000", 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167170+0000", 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7523000000000001e-05 2024-09-18T00:58:40.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167189+0000", 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8530999999999999e-05 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167215+0000", 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6676999999999999e-05 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252445+0000", 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085229416000000002 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252529+0000", 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4740000000000005e-05 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252699+0000", 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016920800000000001 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1680 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.376 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.162068359000003, 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085810110999999994, 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1680 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:40.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167236+0000", 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.048e-06 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167242+0000", 2024-09-18T00:58:40.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6459999999999998e-06 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167259+0000", 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.696e-05 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167271+0000", 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.292e-05 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252839+0000", 2024-09-18T00:58:40.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085567292000000003 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252924+0000", 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5511000000000002e-05 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253045+0000", 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120734 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1681 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.161962262999999, 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085887343000000005, 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1681 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.381 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167342+0000", 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167348+0000", 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4519999999999996e-06 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167372+0000", 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4261999999999999e-05 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167400+0000", 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253091+0000", 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085690895000000003 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253154+0000", 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3262000000000005e-05 2024-09-18T00:58:40.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253228+0000", 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4318000000000002e-05 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1682 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.161884182999998, 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.086061112999999995, 2024-09-18T00:58:40.385 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1682 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168114+0000", 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00069532600000000002 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168123+0000", 2024-09-18T00:58:40.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8319999999999995e-06 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168147+0000", 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3536999999999998e-05 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168163+0000", 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6317e-05 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253274+0000", 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085111390999999995 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253362+0000", 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7384000000000002e-05 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253480+0000", 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011832600000000001 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.389 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1683 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1867776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.161107051999998, 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085456638000000001, 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1683 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169938+0000", 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017419079999999999 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169951+0000", 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2534999999999999e-05 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169990+0000", 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9004999999999997e-05 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170011+0000", 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.156e-05 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253534+0000", 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083522759000000002 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253585+0000", 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1316999999999997e-05 2024-09-18T00:58:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253653+0000", 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7553999999999997e-05 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1684 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.159257416000003, 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083787188999999998, 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1684 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170048+0000", 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.674e-06 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170055+0000", 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3549999999999999e-06 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170073+0000", 2024-09-18T00:58:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7662000000000001e-05 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170087+0000", 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4229e-05 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253701+0000", 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083614346000000006 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253752+0000", 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0760000000000002e-05 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253833+0000", 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1162999999999995e-05 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1685 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.159196618999999, 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083900785000000005, 2024-09-18T00:58:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1685 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170108+0000", 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.229e-06 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170114+0000", 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0079999999999999e-06 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170132+0000", 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7734e-05 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170145+0000", 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2977e-05 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253866+0000", 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083721208000000005 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253934+0000", 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.834e-05 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254007+0000", 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3288999999999996e-05 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1686 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.159141408000004, 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084083526000000006, 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1686 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170163+0000", 2024-09-18T00:58:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1909999999999999e-06 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170180+0000", 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7323e-05 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170198+0000", 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7723e-05 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170212+0000", 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3645999999999999e-05 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254050+0000", 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083838115000000005 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254136+0000", 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6106999999999994e-05 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254245+0000", 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010942099999999999 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1687 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.159073106000001, 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084186574, 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.407 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1687 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170647+0000", 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416649 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170655+0000", 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6510000000000007e-06 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170683+0000", 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7093999999999998e-05 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170701+0000", 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8383e-05 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254283+0000", 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083582091999999997 2024-09-18T00:58:40.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254339+0000", 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6187999999999998e-05 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254417+0000", 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7516999999999997e-05 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1688 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.158577391999998, 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083931026000000006, 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1688 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173553+0000", 2024-09-18T00:58:40.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002827333 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173566+0000", 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3276e-05 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173603+0000", 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6177000000000003e-05 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173628+0000", 2024-09-18T00:58:40.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5343000000000001e-05 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254460+0000", 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.080831745999999996 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254547+0000", 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7045000000000004e-05 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254657+0000", 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011010600000000001 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.554440190000001, 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:40.416 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:40.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.421 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.422 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.422 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.422 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.422 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:40.424 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:40.425 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:40.417+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:40.425 INFO:teuthology.orchestra.run.smithi042.stdout:3377 trunc from 0xcb2f07f to 0x519b94e 2024-09-18T00:58:40.425 INFO:teuthology.orchestra.run.smithi042.stdout:3378 trunc from 0x519b94e to 0x6911c89 2024-09-18T00:58:40.425 INFO:teuthology.orchestra.run.smithi042.stdout:3379 write 0x6024181 thru 0x6030b8e (0xca0e bytes) 2024-09-18T00:58:40.425 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.425 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.425 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.425 INFO:teuthology.orchestra.run.smithi042.stdout:3384 read 0xf7d95f thru 0xf802b5 (0x2957 bytes) 2024-09-18T00:58:40.425 INFO:teuthology.orchestra.run.smithi042.stdout:3385 punch from 0xbccb23 to 0xbd9b05, (0xcfe2 bytes) 2024-09-18T00:58:40.425 INFO:teuthology.orchestra.run.smithi042.stdout:3386 punch from 0x599579e to 0x599f720, (0x9f82 bytes) 2024-09-18T00:58:40.425 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.426 INFO:teuthology.orchestra.run.smithi042.stdout:3388 write 0xa8f92d8 thru 0xa904252 (0xaf7b bytes) 2024-09-18T00:58:40.426 INFO:teuthology.orchestra.run.smithi042.stdout:3390 trunc from 0xa904253 to 0xcd14622 2024-09-18T00:58:40.439 INFO:teuthology.orchestra.run.smithi042.stdout:3391 read 0x47ef576 thru 0x47fbfda (0xca65 bytes) 2024-09-18T00:58:40.441 INFO:teuthology.orchestra.run.smithi042.stdout:3392 writesame 0x47141b4 thru 0x4723e9e data_size 0x1(0xfceb bytes) 2024-09-18T00:58:40.515 INFO:teuthology.orchestra.run.smithi042.stdout:3393 read 0x89b7c93 thru 0x89c6b3d (0xeeab bytes) 2024-09-18T00:58:40.516 INFO:teuthology.orchestra.run.smithi042.stdout:3394 punch from 0x6f22126 to 0x6f23a98, (0x1972 bytes) 2024-09-18T00:58:40.516 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.517 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.524 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:40.517+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:40.527 INFO:teuthology.orchestra.run.smithi042.stdout:3397 trunc from 0xcd14622 to 0x9333c2f 2024-09-18T00:58:40.546 INFO:teuthology.orchestra.run.smithi042.stdout:3400 trunc from 0x9333c2f to 0x8a04e54 2024-09-18T00:58:40.579 INFO:teuthology.orchestra.run.smithi042.stdout:3401 write 0x9531c9e thru 0x9531e99 (0x1fc bytes) 2024-09-18T00:58:40.582 INFO:teuthology.orchestra.run.smithi042.stdout:3402 read 0xc16a68 thru 0xc18d8f (0x2328 bytes) 2024-09-18T00:58:40.583 INFO:teuthology.orchestra.run.smithi042.stdout:3403 punch from 0x8ab0b2b to 0x8ab7596, (0x6a6b bytes) 2024-09-18T00:58:40.583 INFO:teuthology.orchestra.run.smithi042.stdout:3404 read 0x75e0ca8 thru 0x75eab36 (0x9e8f bytes) 2024-09-18T00:58:40.584 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.596 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:40.596 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:40.596 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:40.596 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:40.598 INFO:teuthology.orchestra.run.smithi042.stdout:3407 trunc from 0x9531e9a to 0x4573a74 2024-09-18T00:58:40.612 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:40.617 INFO:teuthology.orchestra.run.smithi042.stdout:3408 punch from 0x120fbf9 to 0x1216a91, (0x6e98 bytes) 2024-09-18T00:58:40.617 INFO:teuthology.orchestra.run.smithi042.stdout:3409 read 0x2318a7a thru 0x231c842 (0x3dc9 bytes) 2024-09-18T00:58:40.625 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:40.617+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:40.642 INFO:teuthology.orchestra.run.smithi042.stdout:3410 write 0x8239900 thru 0x8244500 (0xac01 bytes) 2024-09-18T00:58:40.646 INFO:teuthology.orchestra.run.smithi042.stdout:3411 read 0x758e46b thru 0x758ed57 (0x8ed bytes) 2024-09-18T00:58:40.647 INFO:teuthology.orchestra.run.smithi042.stdout:3412 read 0x132ee21 thru 0x13345e9 (0x57c9 bytes) 2024-09-18T00:58:40.679 INFO:teuthology.orchestra.run.smithi042.stdout:3413 write 0xea1cc38 thru 0xea2a326 (0xd6ef bytes) 2024-09-18T00:58:40.683 INFO:teuthology.orchestra.run.smithi042.stdout:3414 read 0x80368cb thru 0x8040ee7 (0xa61d bytes) 2024-09-18T00:58:40.684 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.684 INFO:teuthology.orchestra.run.smithi042.stdout:3416 writesame 0x2ce89d thru 0x2ce9e6 data_size 0x6(0x14a bytes) 2024-09-18T00:58:40.687 INFO:teuthology.orchestra.run.smithi042.stdout:3417 writesame 0xcf9001c thru 0xcf96873 data_size 0x48(0x6858 bytes) 2024-09-18T00:58:40.691 INFO:teuthology.orchestra.run.smithi042.stdout:3418 punch from 0x1df59e3 to 0x1e013de, (0xb9fb bytes) 2024-09-18T00:58:40.696 INFO:teuthology.orchestra.run.smithi042.stdout:3419 trunc from 0xea2a327 to 0xcfc5106 2024-09-18T00:58:40.711 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.711 INFO:teuthology.orchestra.run.smithi042.stdout:3421 read 0xc827a41 thru 0xc829467 (0x1a27 bytes) 2024-09-18T00:58:40.712 INFO:teuthology.orchestra.run.smithi042.stdout:3422 write 0x4e80967 thru 0x4e81cf4 (0x138e bytes) 2024-09-18T00:58:40.715 INFO:teuthology.orchestra.run.smithi042.stdout:3423 read 0x90c8456 thru 0x90cc976 (0x4521 bytes) 2024-09-18T00:58:40.716 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.716 INFO:teuthology.orchestra.run.smithi042.stdout:3425 writesame 0x3ea18db thru 0x3ea251c data_size 0x3(0xc42 bytes) 2024-09-18T00:58:40.726 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:40.721+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:40.737 INFO:teuthology.orchestra.run.smithi042.stdout:3426 write 0xd61cfd1 thru 0xd62ae19 (0xde49 bytes) 2024-09-18T00:58:40.740 INFO:teuthology.orchestra.run.smithi042.stdout:3428 punch from 0x345d706 to 0x345dea6, (0x7a0 bytes) 2024-09-18T00:58:40.740 INFO:teuthology.orchestra.run.smithi042.stdout:3430 write 0x9c50add thru 0x9c5dc46 (0xd16a bytes) 2024-09-18T00:58:40.744 INFO:teuthology.orchestra.run.smithi042.stdout:3433 punch from 0x7dd1963 to 0x7ddcf69, (0xb606 bytes) 2024-09-18T00:58:40.744 INFO:teuthology.orchestra.run.smithi042.stdout:3434 read 0x154e398 thru 0x154ec9c (0x905 bytes) 2024-09-18T00:58:40.744 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.745 INFO:teuthology.orchestra.run.smithi042.stdout:3436 read 0x607a772 thru 0x607f96c (0x51fb bytes) 2024-09-18T00:58:40.746 INFO:teuthology.orchestra.run.smithi042.stdout:3437 writesame 0x22dd257 thru 0x22e0275 data_size 0x1(0x301f bytes) 2024-09-18T00:58:40.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:58:40.760 INFO:teuthology.orchestra.run.smithi042.stdout:3441 read 0x395b878 thru 0x39630fd (0x7886 bytes) 2024-09-18T00:58:40.772 INFO:teuthology.orchestra.run.smithi042.stdout:3444 write 0xd843dbe thru 0xd850a7c (0xccbf bytes) 2024-09-18T00:58:40.775 INFO:teuthology.orchestra.run.smithi042.stdout:3446 read 0xb7410aa thru 0xb750ff9 (0xff50 bytes) 2024-09-18T00:58:40.776 INFO:teuthology.orchestra.run.smithi042.stdout:3447 punch from 0x1188e3d to 0x119501f, (0xc1e2 bytes) 2024-09-18T00:58:40.776 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.776 INFO:teuthology.orchestra.run.smithi042.stdout:3449 read 0x34c289b thru 0x34c6748 (0x3eae bytes) 2024-09-18T00:58:40.776 INFO:teuthology.orchestra.run.smithi042.stdout:3450 read 0x65c1c41 thru 0x65cd048 (0xb408 bytes) 2024-09-18T00:58:40.782 INFO:teuthology.orchestra.run.smithi042.stdout:3451 trunc from 0xd850a7d to 0xc022267 2024-09-18T00:58:40.785 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:40.786 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:40.786 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:40.786 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:40.786 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:40.796 INFO:teuthology.orchestra.run.smithi042.stdout:3452 read 0x98a58f1 thru 0x98ac14d (0x685d bytes) 2024-09-18T00:58:40.801 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:40.819 INFO:teuthology.orchestra.run.smithi042.stdout:3453 trunc from 0xc022267 to 0x133acca 2024-09-18T00:58:40.827 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:40.821+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:40.874 INFO:teuthology.orchestra.run.smithi042.stdout:3454 write 0xb913cb6 thru 0xb916322 (0x266d bytes) 2024-09-18T00:58:40.879 INFO:teuthology.orchestra.run.smithi042.stdout:3456 read 0x991f8f9 thru 0x99252c2 (0x59ca bytes) 2024-09-18T00:58:40.879 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.879 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.879 INFO:teuthology.orchestra.run.smithi042.stdout:3460 punch from 0xeba96e to 0xec20d5, (0x7767 bytes) 2024-09-18T00:58:40.880 INFO:teuthology.orchestra.run.smithi042.stdout:3462 read 0x6ccaf36 thru 0x6cd8a52 (0xdb1d bytes) 2024-09-18T00:58:40.880 INFO:teuthology.orchestra.run.smithi042.stdout:3463 write 0x2112248 thru 0x211a06f (0x7e28 bytes) 2024-09-18T00:58:40.885 INFO:teuthology.orchestra.run.smithi042.stdout:3464 read 0x66532b4 thru 0x665639d (0x30ea bytes) 2024-09-18T00:58:40.886 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.886 INFO:teuthology.orchestra.run.smithi042.stdout:3466 read 0x6212b23 thru 0x621f8e4 (0xcdc2 bytes) 2024-09-18T00:58:40.887 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:40.887 INFO:teuthology.orchestra.run.smithi042.stdout:3468 writesame 0x4526992 thru 0x452efa7 data_size 0x9(0x8616 bytes) 2024-09-18T00:58:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:3469 trunc from 0xb916323 to 0x3e1503 2024-09-18T00:58:40.929 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:40.925+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:40.940 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:40.940 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:40.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:40.940 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:40.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.940 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:40.940 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.526848764999997, 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:40.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:26797 3.0 3:03a9420d:::rbd_data.117c4c7db1a0.0000000000000071:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:40.946 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.748421966000002, 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073789632999999993, 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 26797 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187640+0000", 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187653+0000", 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2357e-05 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187684+0000", 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1582999999999999e-05 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187699+0000", 2024-09-18T00:58:40.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5003e-05 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187806+0000", 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106776 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.188026+0000", 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021975799999999999 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7", 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.201663+0000", 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013637768 2024-09-18T00:58:40.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261226+0000", 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059562916 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261261+0000", 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4360000000000003e-05 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:40.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261354+0000", 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3041999999999995e-05 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261427+0000", 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3553000000000005e-05 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64346 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:40.952 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.34717038, 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048905196999999997, 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64346 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588890+0000", 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3e-07 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588893+0000", 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.106e-06 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588904+0000", 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1037e-05 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588911+0000", 2024-09-18T00:58:40.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2729999999999997e-06 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637426+0000", 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048514483999999997 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637537+0000", 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110901 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637795+0000", 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025776599999999998 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64347 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.347138507, 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049469502999999998, 2024-09-18T00:58:40.957 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64347 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588922+0000", 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4300000000000003e-07 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588925+0000", 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T00:58:40.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588934+0000", 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0329999999999997e-06 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588946+0000", 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1547e-05 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637872+0000", 2024-09-18T00:58:40.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048926202000000002 2024-09-18T00:58:40.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638047+0000", 2024-09-18T00:58:40.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174566 2024-09-18T00:58:40.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638391+0000", 2024-09-18T00:58:40.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034401499999999998 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64348 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.347097636999999, 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050239487999999999, 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.962 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64348 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:40.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588963+0000", 2024-09-18T00:58:40.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.133e-06 2024-09-18T00:58:40.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588969+0000", 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4229999999999996e-06 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588996+0000", 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7396e-05 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589163+0000", 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016677900000000001 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638498+0000", 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049334743 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638715+0000", 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000217414 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639202+0000", 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048660000000000001 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.966 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64349 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346868732000001, 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050607656000000001, 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.967 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64349 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589192+0000", 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.536e-06 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589201+0000", 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-06 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589222+0000", 2024-09-18T00:58:40.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589241+0000", 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8960000000000001e-05 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639285+0000", 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050043892999999999 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639457+0000", 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017286299999999999 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639799+0000", 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034117400000000002 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64350 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:40.971 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346798387, 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051028502000000003, 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64350 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:40.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589262+0000", 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1570000000000001e-06 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589280+0000", 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7943000000000001e-05 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589299+0000", 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8940000000000002e-05 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589316+0000", 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6654e-05 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639872+0000", 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050556151000000001 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639988+0000", 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115423 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640290+0000", 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030223400000000002 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.975 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64351 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346722647, 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051389975999999997, 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.976 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64351 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589338+0000", 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589345+0000", 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589363+0000", 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8346e-05 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589381+0000", 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7357e-05 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640373+0000", 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050992347 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640504+0000", 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000131011 2024-09-18T00:58:40.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640727+0000", 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022310799999999999 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64352 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346659118, 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051932649999999997, 2024-09-18T00:58:40.980 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64352 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589402+0000", 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.237e-06 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589408+0000", 2024-09-18T00:58:40.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2770000000000002e-06 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589426+0000", 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8006e-05 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589441+0000", 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4934e-05 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640803+0000", 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051361619999999997 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640986+0000", 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018303199999999999 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641333+0000", 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034754399999999998 2024-09-18T00:58:40.984 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64353 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346600724, 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052565148999999999, 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.985 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64353 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589460+0000", 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4300000000000001e-06 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589466+0000", 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.446e-06 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589485+0000", 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8726999999999999e-05 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589499+0000", 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.402e-05 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641427+0000", 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051927945000000003 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641601+0000", 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174859 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642024+0000", 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000422722 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64354 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346526024999999, 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053055607999999997, 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64354 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589535+0000", 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3430000000000001e-06 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589542+0000", 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589561+0000", 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9207000000000001e-05 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589577+0000", 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5933000000000001e-05 2024-09-18T00:58:40.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642111+0000", 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052533799999999999 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642271+0000", 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015998499999999999 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642589+0000", 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031818999999999999 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64355 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346461989, 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053570327000000001, 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.994 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64355 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589599+0000", 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589605+0000", 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8830000000000003e-06 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589622+0000", 2024-09-18T00:58:40.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.732e-05 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589636+0000", 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3703000000000001e-05 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642672+0000", 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053036204000000003 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:40.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642828+0000", 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155451 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643168+0000", 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034051900000000001 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64356 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:40.998 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346404476, 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054203299000000003, 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64356 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:40.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589656+0000", 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589662+0000", 2024-09-18T00:58:41.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9800000000000003e-06 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589681+0000", 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8300000000000001e-05 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589695+0000", 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3997e-05 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:41.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643249+0000", 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053554101999999999 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643423+0000", 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174062 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643859+0000", 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043562499999999998 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64357 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346346526, 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054239849, 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64357 2024-09-18T00:58:41.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:41.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589714+0000", 2024-09-18T00:58:41.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.266e-06 2024-09-18T00:58:41.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:41.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589720+0000", 2024-09-18T00:58:41.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2340000000000002e-06 2024-09-18T00:58:41.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:41.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589738+0000", 2024-09-18T00:58:41.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7932999999999999e-05 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589752+0000", 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4867e-05 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643934+0000", 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054181721000000002 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643953+0000", 2024-09-18T00:58:41.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8828e-05 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64358 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346288647, 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054261125, 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:41.007 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64358 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589772+0000", 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1170000000000001e-06 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589778+0000", 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8429999999999998e-06 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:41.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589799+0000", 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0727e-05 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589813+0000", 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4403000000000001e-05 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644021+0000", 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054207784000000002 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644032+0000", 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1251e-05 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64359 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346226680999999, 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286586999999997, 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:41.011 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64359 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:41.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589834+0000", 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589841+0000", 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.956e-06 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589864+0000", 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3124000000000001e-05 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589879+0000", 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.486e-05 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644099+0000", 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054219429 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644120+0000", 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1004000000000001e-05 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64360 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346162487000001, 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054290699999999997, 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:41.015 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64360 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589898+0000", 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3600000000000002e-07 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589903+0000", 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4669999999999998e-06 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589913+0000", 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0463e-05 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589921+0000", 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7959999999999996e-06 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644173+0000", 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054251903999999997 2024-09-18T00:58:41.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644188+0000", 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5133999999999999e-05 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64361 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346129008, 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054320101000000003, 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:41.019 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64361 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5000000000000002e-07 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589935+0000", 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2799999999999999e-06 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589944+0000", 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589954+0000", 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0036e-05 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644240+0000", 2024-09-18T00:58:41.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286273000000003 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644251+0000", 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0392e-05 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64362 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346095862, 2024-09-18T00:58:41.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054340196, 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64362 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:41.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589978+0000", 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3570000000000001e-05 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589991+0000", 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3196999999999999e-05 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590000+0000", 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2800000000000003e-06 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644294+0000", 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054294873 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644304+0000", 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6560000000000004e-06 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64363 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:41.027 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.346049832, 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054356660000000001, 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64363 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:41.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590011+0000", 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7400000000000003e-07 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590014+0000", 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590024+0000", 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5670000000000001e-06 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590031+0000", 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3830000000000003e-06 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644354+0000", 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054323429999999999 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644367+0000", 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.217e-05 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:41.031 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:41.034 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:41.034 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:41.029+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:41.034 INFO:teuthology.orchestra.run.smithi042.stdout:3470 writesame 0xce027e5 thru 0xce0a4f5 data_size 0x1(0x7d11 bytes) 2024-09-18T00:58:41.034 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.034 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.034 INFO:teuthology.orchestra.run.smithi042.stdout:3476 read 0x9856502 thru 0x9858e1f (0x291e bytes) 2024-09-18T00:58:41.034 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.035 INFO:teuthology.orchestra.run.smithi042.stdout:3479 read 0xb7d7a7a thru 0xb7e2c42 (0xb1c9 bytes) 2024-09-18T00:58:41.035 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.035 INFO:teuthology.orchestra.run.smithi042.stdout:3481 read 0x7b5ba2b thru 0x7b66c1d (0xb1f3 bytes) 2024-09-18T00:58:41.035 INFO:teuthology.orchestra.run.smithi042.stdout:3482 trunc from 0xce0a4f6 to 0x4204561 2024-09-18T00:58:41.035 INFO:teuthology.orchestra.run.smithi042.stdout:3483 write 0x5cafc28 thru 0x5cbfb49 (0xff22 bytes) 2024-09-18T00:58:41.035 INFO:teuthology.orchestra.run.smithi042.stdout:3484 writesame 0xa0af885 thru 0xa0b6a89 data_size 0x11(0x7205 bytes) 2024-09-18T00:58:41.049 INFO:teuthology.orchestra.run.smithi042.stdout:3485 writesame 0xe81378f thru 0xe81b931 data_size 0xb(0x81a3 bytes) 2024-09-18T00:58:41.049 INFO:teuthology.orchestra.run.smithi042.stdout:3486 read 0x48391f6 thru 0x48447f6 (0xb601 bytes) 2024-09-18T00:58:41.050 INFO:teuthology.orchestra.run.smithi042.stdout:3487 punch from 0x8096477 to 0x8098c2d, (0x27b6 bytes) 2024-09-18T00:58:41.050 INFO:teuthology.orchestra.run.smithi042.stdout:3488 read 0x2a4fdb0 thru 0x2a53f85 (0x41d6 bytes) 2024-09-18T00:58:41.050 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.050 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.050 INFO:teuthology.orchestra.run.smithi042.stdout:3491 read 0xcb2abb6 thru 0xcb372d0 (0xc71b bytes) 2024-09-18T00:58:41.050 INFO:teuthology.orchestra.run.smithi042.stdout:3492 write 0xc74987e thru 0xc753188 (0x990b bytes) 2024-09-18T00:58:41.053 INFO:teuthology.orchestra.run.smithi042.stdout:3494 punch from 0x232a73c to 0x2335523, (0xade7 bytes) 2024-09-18T00:58:41.053 INFO:teuthology.orchestra.run.smithi042.stdout:3495 write 0xcf75a9c thru 0xcf797fb (0x3d60 bytes) 2024-09-18T00:58:41.059 INFO:teuthology.orchestra.run.smithi042.stdout:3497 writesame 0x62fcf73 thru 0x630898b data_size 0x3d(0xba19 bytes) 2024-09-18T00:58:41.067 INFO:teuthology.orchestra.run.smithi042.stdout:3500 punch from 0xbab0458 to 0xbab4fef, (0x4b97 bytes) 2024-09-18T00:58:41.089 INFO:teuthology.orchestra.run.smithi042.stdout:3501 trunc from 0xe81b932 to 0x3a60bd3 2024-09-18T00:58:41.124 INFO:teuthology.orchestra.run.smithi042.stdout:3504 writesame 0x7fc216c thru 0x7fcca6a data_size 0xab(0xa8ff bytes) 2024-09-18T00:58:41.125 INFO:teuthology.orchestra.run.smithi042.stdout:3505 write 0x6a6b79b thru 0x6a72d6a (0x75d0 bytes) 2024-09-18T00:58:41.134 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:41.129+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:41.141 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:41.152 INFO:teuthology.orchestra.run.smithi042.stdout:3506 writesame 0xcb64bc7 thru 0xcb6cf5e data_size 0x4(0x8398 bytes) 2024-09-18T00:58:41.155 INFO:teuthology.orchestra.run.smithi042.stdout:3507 punch from 0x66a1cb7 to 0x66a28c1, (0xc0a bytes) 2024-09-18T00:58:41.171 INFO:teuthology.orchestra.run.smithi042.stdout:3510 trunc from 0xcb6cf5f to 0x425daef 2024-09-18T00:58:41.185 INFO:teuthology.orchestra.run.smithi042.stdout:3511 read 0x1a6be15 thru 0x1a76e05 (0xaff1 bytes) 2024-09-18T00:58:41.186 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.186 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.198 INFO:teuthology.orchestra.run.smithi042.stdout:3516 write 0x4a70c61 thru 0x4a7c50f (0xb8af bytes) 2024-09-18T00:58:41.200 INFO:teuthology.orchestra.run.smithi042.stdout:3517 punch from 0x3e2ea3e to 0x3e2fa80, (0x1042 bytes) 2024-09-18T00:58:41.217 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:41.217 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:41.217 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:41.217 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:41.220 INFO:teuthology.orchestra.run.smithi042.stdout:3518 writesame 0x9231305 thru 0x92403ac data_size 0x97(0xf0a8 bytes) 2024-09-18T00:58:41.220 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.220 INFO:teuthology.orchestra.run.smithi042.stdout:3522 read 0x30f0e51 thru 0x30fee74 (0xe024 bytes) 2024-09-18T00:58:41.232 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:41.234 INFO:teuthology.orchestra.run.smithi042.stdout:3524 writesame 0xa71d57d thru 0xa7248fe data_size 0x5(0x7382 bytes) 2024-09-18T00:58:41.234 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:41.229+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:41.236 INFO:teuthology.orchestra.run.smithi042.stdout:3525 write 0x60b288c thru 0x60ba48e (0x7c03 bytes) 2024-09-18T00:58:41.239 INFO:teuthology.orchestra.run.smithi042.stdout:3526 write 0x392c328 thru 0x393b182 (0xee5b bytes) 2024-09-18T00:58:41.243 INFO:teuthology.orchestra.run.smithi042.stdout:3527 read 0x2e39d9a thru 0x2e49c26 (0xfe8d bytes) 2024-09-18T00:58:41.244 INFO:teuthology.orchestra.run.smithi042.stdout:3528 read 0x170d983 thru 0x1717ba5 (0xa223 bytes) 2024-09-18T00:58:41.257 INFO:teuthology.orchestra.run.smithi042.stdout:3529 write 0xcf38c54 thru 0xcf408fa (0x7ca7 bytes) 2024-09-18T00:58:41.263 INFO:teuthology.orchestra.run.smithi042.stdout:3531 trunc from 0xcf408fb to 0xb74dd2a 2024-09-18T00:58:41.286 INFO:teuthology.orchestra.run.smithi042.stdout:3532 trunc from 0xb74dd2a to 0x481dbf8 2024-09-18T00:58:41.298 INFO:teuthology.orchestra.run.smithi042.stdout:3533 punch from 0x96adfb to 0x973076, (0x827b bytes) 2024-09-18T00:58:41.298 INFO:teuthology.orchestra.run.smithi042.stdout:3534 read 0x1d18a02 thru 0x1d2284e (0x9e4d bytes) 2024-09-18T00:58:41.314 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:41.314 INFO:teuthology.orchestra.run.smithi042.stdout:3535 write 0x9c13973 thru 0x9c149a1 (0x102f bytes) 2024-09-18T00:58:41.314 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:41.316 INFO:teuthology.orchestra.run.smithi042.stdout:3536 write 0x25ad3bf thru 0x25b973b (0xc37d bytes) 2024-09-18T00:58:41.318 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.318 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.318 INFO:teuthology.orchestra.run.smithi042.stdout:3539 punch from 0x93b7e8e to 0x93c59a1, (0xdb13 bytes) 2024-09-18T00:58:41.318 INFO:teuthology.orchestra.run.smithi042.stdout:3540 write 0x2cd5d82 thru 0x2cdeeca (0x9149 bytes) 2024-09-18T00:58:41.328 INFO:teuthology.orchestra.run.smithi042.stdout:3542 trunc from 0x9c149a2 to 0x4111e77 2024-09-18T00:58:41.333 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:41.333 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:41.335 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:41.329+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:41.339 INFO:teuthology.orchestra.run.smithi042.stdout:3543 read 0x2620793 thru 0x262ca7b (0xc2e9 bytes) 2024-09-18T00:58:41.340 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.350 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:41.351 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:41.356 INFO:teuthology.orchestra.run.smithi042.stdout:3546 write 0x908786b thru 0x9094ee7 (0xd67d bytes) 2024-09-18T00:58:41.362 INFO:teuthology.orchestra.run.smithi042.stdout:3547 write 0x52770d3 thru 0x527715a (0x88 bytes) 2024-09-18T00:58:41.365 INFO:teuthology.orchestra.run.smithi042.stdout:3548 trunc from 0x9094ee8 to 0x98f1382 2024-09-18T00:58:41.367 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:41.368 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:41.374 INFO:teuthology.orchestra.run.smithi042.stdout:3551 write 0x3743f9 thru 0x37bd97 (0x799f bytes) 2024-09-18T00:58:41.375 INFO:teuthology.orchestra.run.smithi042.stdout:3552 write 0x6800495 thru 0x68060a4 (0x5c10 bytes) 2024-09-18T00:58:41.378 INFO:teuthology.orchestra.run.smithi042.stdout:3553 read 0x7244a36 thru 0x7252603 (0xdbce bytes) 2024-09-18T00:58:41.378 INFO:teuthology.orchestra.run.smithi042.stdout:3555 read 0x517dca thru 0x521051 (0x9288 bytes) 2024-09-18T00:58:41.388 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:41.388 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:41.393 INFO:teuthology.orchestra.run.smithi042.stdout:3556 writesame 0xd3acc38 thru 0xd3b67d2 data_size 0x1f(0x9b9b bytes) 2024-09-18T00:58:41.396 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.396 INFO:teuthology.orchestra.run.smithi042.stdout:3559 writesame 0x210f89 thru 0x214f38 data_size 0x8(0x3fb0 bytes) 2024-09-18T00:58:41.399 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:41.399 INFO:teuthology.orchestra.run.smithi042.stdout:3561 read 0x86c6d81 thru 0x86cd785 (0x6a05 bytes) 2024-09-18T00:58:41.400 INFO:teuthology.orchestra.run.smithi042.stdout:3563 writesame 0x4f82dd6 thru 0x4f84243 data_size 0x5(0x146e bytes) 2024-09-18T00:58:41.402 INFO:teuthology.orchestra.run.smithi042.stdout:3565 write 0x9e5f41f thru 0x9e650fe (0x5ce0 bytes) 2024-09-18T00:58:41.408 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:41.411 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:41.411 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:41.411 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:41.411 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:41.411 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:41.420 INFO:teuthology.orchestra.run.smithi042.stdout:3566 trunc from 0xd3b67d3 to 0x2831ce3 2024-09-18T00:58:41.427 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:41.428 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:41.428 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:41.436 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:41.429+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:41.437 INFO:teuthology.orchestra.run.smithi042.stdout:3567 read 0x1ec3b62 thru 0x1ec8fe4 (0x5483 bytes) 2024-09-18T00:58:41.440 INFO:teuthology.orchestra.run.smithi042.stdout:3568 trunc from 0x2831ce3 to 0xf992e8 2024-09-18T00:58:41.449 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:41.449 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:41.470 INFO:teuthology.orchestra.run.smithi042.stdout:3569 write 0x3155924 thru 0x315a5c7 (0x4ca4 bytes) 2024-09-18T00:58:41.486 INFO:teuthology.orchestra.run.smithi042.stdout:3570 write 0x5dbc3fc thru 0x5dbdcb9 (0x18be bytes) 2024-09-18T00:58:41.494 INFO:teuthology.orchestra.run.smithi042.stdout:3572 trunc from 0x5dbdcba to 0x7aa4f9c 2024-09-18T00:58:41.502 INFO:teuthology.orchestra.run.smithi042.stdout:3574 writesame 0xd3b808 thru 0xd41570 data_size 0x3(0x5d69 bytes) 2024-09-18T00:58:41.516 INFO:teuthology.orchestra.run.smithi042.stdout:3575 trunc from 0x7aa4f9c to 0xc9661cb 2024-09-18T00:58:41.527 INFO:teuthology.orchestra.run.smithi042.stdout:3577 writesame 0xa85c4a9 thru 0xa862139 data_size 0x3(0x5c91 bytes) 2024-09-18T00:58:41.537 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:41.533+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:41.541 INFO:teuthology.orchestra.run.smithi042.stdout:3578 writesame 0x59c61d1 thru 0x59c6399 data_size 0x1(0x1c9 bytes) 2024-09-18T00:58:41.545 INFO:teuthology.orchestra.run.smithi042.stdout:3579 write 0xa23fa17 thru 0xa247a9d (0x8087 bytes) 2024-09-18T00:58:41.553 INFO:teuthology.orchestra.run.smithi042.stdout:3580 writesame 0x2906672 thru 0x2911c3b data_size 0x1(0xb5ca bytes) 2024-09-18T00:58:41.558 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:41.558 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:41.573 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:41.618 INFO:teuthology.orchestra.run.smithi042.stdout:3581 writesame 0xd0fed56 thru 0xd10e555 data_size 0x40(0xf800 bytes) 2024-09-18T00:58:41.637 INFO:teuthology.orchestra.run.smithi042.stdout:3582 trunc from 0xd10e556 to 0x66e4286 2024-09-18T00:58:41.654 INFO:teuthology.orchestra.run.smithi042.stdout:3583 writesame 0x27df158 thru 0x27e5e5e data_size 0x13(0x6d07 bytes) 2024-09-18T00:58:41.665 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:41.661+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:41.667 INFO:teuthology.orchestra.run.smithi042.stdout:3584 trunc from 0x66e4286 to 0x9d26387 2024-09-18T00:58:41.679 INFO:teuthology.orchestra.run.smithi042.stdout:3585 clone 28 order 18 su 2048 sc 10 2024-09-18T00:58:41.739 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:41.733+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:41.742 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:58:41.742 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:58:41.742 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:58:41.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-18T00:58:41.746 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:41.746 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:41.751 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:58:41.762 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:41.849 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:41.845+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:41.928 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:41.928 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:41.941 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:41.933+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:41.943 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:41.992 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-18T00:58:42.003 INFO:teuthology.orchestra.run.smithi042.stdout:3586 read 0x6ef62ef thru 0x6f00756 (0xa468 bytes) 2024-09-18T00:58:42.041 INFO:teuthology.orchestra.run.smithi042.stdout:3587 writesame 0x2e6a2f0 thru 0x2e75277 data_size 0x8(0xaf88 bytes) 2024-09-18T00:58:42.041 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:42.037+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:42.098 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-18T00:58:42.093+0000 7fdb35eeb640 -1 mon.a@0(leader).osd e72 definitely_dead 0 2024-09-18T00:58:42.138 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:42.138 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:42.142 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:42.137+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:42.156 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:42.243 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:42.237+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:42.293 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:42.294 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:42.309 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:42.358 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:42.353+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:42.445 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:42.441+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:42.447 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:42.447 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:42.462 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:42.550 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:42.545+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:42.604 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:42.604 INFO:teuthology.orchestra.run.smithi042.stdout:3590 read 0x4962580 thru 0x49720ba (0xfb3b bytes) 2024-09-18T00:58:42.636 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:42.636 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:42.646 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:42.641+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:42.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:42.747 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:42.741+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:42.754 INFO:teuthology.orchestra.run.smithi042.stderr:marked out osd.2. 2024-09-18T00:58:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:42.847 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:42.841+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:42.857 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:42.948 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:42.941+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:43.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:43.050 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:43.045+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:43.150 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:43.145+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:43.202 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:43.202 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:43.217 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:43.251 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:43.245+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:43.352 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:43.345+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:43.381 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:43.381 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:43.395 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:43.453 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:43.449+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:43.470 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:43.554 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:43.549+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:43.567 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:43.567 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:43.582 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:43.626 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:43.626 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:43.644 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:43.644 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:43.654 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:43.649+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:43.661 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:43.661 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:43.678 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:43.678 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:43.699 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:43.699 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:43.720 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:43.721 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:43.742 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:43.742 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:43.743 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:43.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.665007385999999, 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:58:43.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.748 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.664077794999997, 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:58:43.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.664063734999999, 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.753 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:58:43.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:58:43.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:58:43.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:58:43.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:58:43.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.757 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.66277316, 2024-09-18T00:58:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:58:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:58:43.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.759 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:43.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:58:43.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:58:43.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:58:43.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:58:43.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:58:43.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:58:43.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.662752687999998, 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:58:43.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:58:43.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.659958234999998, 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:58:43.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:58:43.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:58:43.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:58:43.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:58:43.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:58:43.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.659943050999999, 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:58:43.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.774 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:43.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:58:43.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:58:43.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:58:43.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:58:43.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.659920104999998, 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:58:43.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:58:43.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:43.782 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.655742466, 2024-09-18T00:58:43.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:58:43.783 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:58:43.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.783 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:58:43.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:58:43.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:58:43.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:58:43.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:58:43.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:58:43.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:58:43.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:58:43.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:58:43.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:58:43.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:58:43.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:58:43.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:58:43.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.787 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.655721249000003, 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:58:43.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.655699865999999, 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:58:43.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:58:43.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.651800012000002, 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.797 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:58:43.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.651785811000003, 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:58:43.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:58:43.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.651767774, 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:58:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.810 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.651753708000001, 2024-09-18T00:58:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:58:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:58:43.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:58:43.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.647728356999998, 2024-09-18T00:58:43.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:43.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:58:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:58:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:58:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:58:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:58:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:58:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:58:43.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:58:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:58:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:58:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:58:43.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.647714344999997, 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:58:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:58:43.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:58:43.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.647700381, 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:58:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:58:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:58:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:58:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:58:43.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.830 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 54.885842427, 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:58:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:58:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:58:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:58:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:58:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:58:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:58:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:58:43.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:58:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:58:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:58:43.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:58:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:58:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:58:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:58:43.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:58:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:58:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:58:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.760366056000002, 2024-09-18T00:58:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:58:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:43.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:58:43.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:58:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:58:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:58:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:58:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:58:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:58:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:58:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:58:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:43.845 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:43.845 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:43.845 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:58:43.847 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:43.847 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:43.848 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:43.848 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:43.841+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:43.948 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:43.941+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:44.021 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:44.021 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:44.021 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:44.022 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:44.037 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:44.078 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:44.073+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:44.148 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:44.145+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:44.193 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:44.193 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:44.194 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:44.194 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:44.194 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:44.208 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:44.249 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:44.249+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:44.329 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:44.329 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:44.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:44.329 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:44.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.329 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:58:44.329 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:44.329 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 77.011361893, 2024-09-18T00:58:44.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101264911, 2024-09-18T00:58:44.329 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4226", 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2155402750", 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313760+0000", 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.612e-06 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313768+0000", 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9249999999999995e-06 2024-09-18T00:58:44.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313786+0000", 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8253999999999998e-05 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313803+0000", 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313935+0000", 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013180399999999999 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313954+0000", 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8768999999999999e-05 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.411874+0000", 2024-09-18T00:58:44.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.097920646 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412292+0000", 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041777699999999998 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412675+0000", 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038262300000000001 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:44.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414019+0000", 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013436699999999999 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414872+0000", 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085320999999999997 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414955+0000", 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3405999999999994e-05 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.415023+0000", 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7825999999999997e-05 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15500 3.11 3:8ed6a97b:::rbd_data.117c53e529c1.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.510601872999999, 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046818956000000002, 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.336 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15500 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:44.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814521+0000", 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.977e-06 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814536+0000", 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.487e-05 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814572+0000", 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5567e-05 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814588+0000", 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6546e-05 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814675+0000", 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6675999999999995e-05 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814839+0000", 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016414199999999999 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.819908+0000", 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050695280000000002 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861139+0000", 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041230771999999999 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861208+0000", 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8722999999999999e-05 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861277+0000", 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9246999999999996e-05 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861337+0000", 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9908000000000003e-05 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.341 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26860 3.a 3:523ff797:::rbd_data.117c4c7db1a0.0000000000000039:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.952427753999999, 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028475341000000001, 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.342 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26860 2024-09-18T00:58:44.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.343 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:44.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372692+0000", 2024-09-18T00:58:44.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372693+0000", 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0550000000000001e-06 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372699+0000", 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999996e-06 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372714+0000", 2024-09-18T00:58:44.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4673e-05 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372722+0000", 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4659999999999999e-06 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372787+0000", 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5288000000000002e-05 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.372874+0000", 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6485999999999998e-05 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.377992+0000", 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0051177159999999996 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401008+0000", 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.02301603 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:44.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401026+0000", 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8712e-05 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401082+0000", 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5648e-05 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.401168+0000", 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5476999999999995e-05 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64289 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 3227648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.738738058999999, 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025443502999999999, 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64289 2024-09-18T00:58:44.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586382+0000", 2024-09-18T00:58:44.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.84e-07 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586385+0000", 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2060000000000001e-06 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586392+0000", 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.586396+0000", 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5859999999999998e-06 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611580+0000", 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025183646000000001 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611667+0000", 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7350999999999998e-05 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.611825+0000", 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015804999999999999 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64403 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 475136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737791575999999, 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025296714000000001, 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.352 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64403 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587328+0000", 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587329+0000", 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3700000000000001e-07 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587331+0000", 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9889999999999999e-06 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587338+0000", 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5209999999999997e-06 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587343+0000", 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612401+0000", 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025057505000000001 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612478+0000", 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6817999999999995e-05 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612625+0000", 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014740099999999999 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64404 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 655360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737770289, 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025540833999999998, 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64404 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:44.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:44.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587350+0000", 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587352+0000", 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587362+0000", 2024-09-18T00:58:44.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0125e-05 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587367+0000", 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1150000000000002e-06 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612671+0000", 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025303638 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612751+0000", 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0107999999999999e-05 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612891+0000", 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013946 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.361 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64405 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737744933999998, 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025814888000000001, 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64405 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.362 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587375+0000", 2024-09-18T00:58:44.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4000000000000002e-07 2024-09-18T00:58:44.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587379+0000", 2024-09-18T00:58:44.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1439999999999999e-06 2024-09-18T00:58:44.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587398+0000", 2024-09-18T00:58:44.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587405+0000", 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3740000000000003e-06 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.612931+0000", 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025526604000000001 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613008+0000", 2024-09-18T00:58:44.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6508000000000007e-05 2024-09-18T00:58:44.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613190+0000", 2024-09-18T00:58:44.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018205200000000001 2024-09-18T00:58:44.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.366 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.367 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64406 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.367 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:44.367 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737692934000002, 2024-09-18T00:58:44.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026015033, 2024-09-18T00:58:44.367 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.367 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.367 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.367 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.367 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.368 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.368 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64406 2024-09-18T00:58:44.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.368 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:44.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587427+0000", 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587428+0000", 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0399999999999996e-07 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587431+0000", 2024-09-18T00:58:44.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1870000000000001e-06 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587441+0000", 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0349999999999999e-05 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587447+0000", 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.083e-06 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613236+0000", 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025788953 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613288+0000", 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2275999999999999e-05 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613442+0000", 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015358 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.371 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.372 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.372 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64407 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.372 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:44.372 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737663935, 2024-09-18T00:58:44.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026326130999999999, 2024-09-18T00:58:44.372 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.372 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.373 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.373 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.373 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.373 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.373 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64407 2024-09-18T00:58:44.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.373 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587456+0000", 2024-09-18T00:58:44.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:58:44.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587459+0000", 2024-09-18T00:58:44.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3750000000000001e-06 2024-09-18T00:58:44.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587465+0000", 2024-09-18T00:58:44.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7150000000000003e-06 2024-09-18T00:58:44.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587469+0000", 2024-09-18T00:58:44.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8060000000000002e-06 2024-09-18T00:58:44.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613488+0000", 2024-09-18T00:58:44.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026018645999999999 2024-09-18T00:58:44.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613576+0000", 2024-09-18T00:58:44.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7892000000000005e-05 2024-09-18T00:58:44.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613782+0000", 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000206233 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64408 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737643691999999, 2024-09-18T00:58:44.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026568468000000001, 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64408 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587476+0000", 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587477+0000", 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7500000000000001e-07 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587479+0000", 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1900000000000002e-06 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587487+0000", 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1869999999999997e-06 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587492+0000", 2024-09-18T00:58:44.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3149999999999999e-06 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613823+0000", 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026330339000000001 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.613885+0000", 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2391999999999995e-05 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614045+0000", 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015966999999999999 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64409 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737620714999998, 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026785499000000001, 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.383 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64409 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587499+0000", 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587500+0000", 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587502+0000", 2024-09-18T00:58:44.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9420000000000002e-06 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587508+0000", 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1840000000000004e-06 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587513+0000", 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9640000000000002e-06 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614075+0000", 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026561889000000002 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614124+0000", 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9177e-05 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614285+0000", 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016103100000000001 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.388 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64410 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.388 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:44.388 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737600766, 2024-09-18T00:58:44.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026987559000000001, 2024-09-18T00:58:44.388 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.388 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64410 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587519+0000", 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587520+0000", 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9200000000000002e-07 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587522+0000", 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1500000000000002e-06 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587529+0000", 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5839999999999997e-06 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587534+0000", 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.826e-06 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614317+0000", 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026783065000000002 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614385+0000", 2024-09-18T00:58:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7682999999999996e-05 2024-09-18T00:58:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614507+0000", 2024-09-18T00:58:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121859 2024-09-18T00:58:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64411 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737579280999999, 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027224245000000001, 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64411 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587541+0000", 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587543+0000", 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587550+0000", 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5810000000000002e-06 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587555+0000", 2024-09-18T00:58:44.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0490000000000003e-06 2024-09-18T00:58:44.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614543+0000", 2024-09-18T00:58:44.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.026988056 2024-09-18T00:58:44.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614619+0000", 2024-09-18T00:58:44.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5932000000000004e-05 2024-09-18T00:58:44.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614765+0000", 2024-09-18T00:58:44.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014603899999999999 2024-09-18T00:58:44.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.397 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.398 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64412 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2097152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.398 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:44.398 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737558536000002, 2024-09-18T00:58:44.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027478605, 2024-09-18T00:58:44.399 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.399 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.399 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64412 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587561+0000", 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587562+0000", 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6499999999999999e-07 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587564+0000", 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3300000000000001e-06 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587571+0000", 2024-09-18T00:58:44.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2949999999999999e-06 2024-09-18T00:58:44.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587575+0000", 2024-09-18T00:58:44.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6360000000000002e-06 2024-09-18T00:58:44.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614798+0000", 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027222860000000002 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.614874+0000", 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5383000000000007e-05 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615040+0000", 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016653600000000001 2024-09-18T00:58:44.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64413 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737537732, 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027709421000000001, 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.404 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.405 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64413 2024-09-18T00:58:44.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.405 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:44.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587582+0000", 2024-09-18T00:58:44.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587583+0000", 2024-09-18T00:58:44.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587585+0000", 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587592+0000", 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9700000000000002e-06 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587597+0000", 2024-09-18T00:58:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8289999999999998e-06 2024-09-18T00:58:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615071+0000", 2024-09-18T00:58:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027474218000000002 2024-09-18T00:58:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615143+0000", 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1505000000000001e-05 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615292+0000", 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014900199999999999 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64414 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2457600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737516194000001, 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027920637000000002, 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64414 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:44.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:44.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587604+0000", 2024-09-18T00:58:44.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.08e-07 2024-09-18T00:58:44.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587606+0000", 2024-09-18T00:58:44.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1299999999999999e-06 2024-09-18T00:58:44.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587612+0000", 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1319999999999999e-06 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587618+0000", 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.147e-06 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615323+0000", 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027705423 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615366+0000", 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2914000000000003e-05 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615524+0000", 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015848300000000001 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64415 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2637824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737491873, 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028241265000000002, 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64415 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:44.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:44.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587628+0000", 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587631+0000", 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5550000000000001e-06 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587639+0000", 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.807e-06 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587644+0000", 2024-09-18T00:58:44.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2630000000000003e-06 2024-09-18T00:58:44.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615633+0000", 2024-09-18T00:58:44.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027989342 2024-09-18T00:58:44.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615703+0000", 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9956000000000006e-05 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615869+0000", 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016597500000000001 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64416 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2818048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737468576000001, 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028575605, 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64416 2024-09-18T00:58:44.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587651+0000", 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587652+0000", 2024-09-18T00:58:44.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-18T00:58:44.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587655+0000", 2024-09-18T00:58:44.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-18T00:58:44.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587662+0000", 2024-09-18T00:58:44.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7789999999999996e-06 2024-09-18T00:58:44.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587668+0000", 2024-09-18T00:58:44.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2109999999999997e-06 2024-09-18T00:58:44.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.615916+0000", 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028248412000000001 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616009+0000", 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2682999999999993e-05 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616227+0000", 2024-09-18T00:58:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218345 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64417 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737444565000001, 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028891495, 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64417 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587675+0000", 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587676+0000", 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.34e-07 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587683+0000", 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8739999999999997e-06 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587692+0000", 2024-09-18T00:58:44.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2159999999999995e-06 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587698+0000", 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.716e-06 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616259+0000", 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028561733999999998 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616369+0000", 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110104 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616567+0000", 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000197417 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.429 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:64418 3.d 3:b6cf25c2:::rbd_data.117c689d3abc.0000000000000005:17 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.737414121, 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029401729000000001, 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 64418 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:44.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587706+0000", 2024-09-18T00:58:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8400000000000004e-07 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587709+0000", 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5390000000000002e-06 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587717+0000", 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0819999999999999e-06 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.587722+0000", 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4140000000000002e-06 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616869+0000", 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.029146763999999999 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.616926+0000", 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7250000000000002e-05 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:44.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:44.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:25.617108+0000", 2024-09-18T00:58:44.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181096 2024-09-18T00:58:44.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.434 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:44.434 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:44.434 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:44.436 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:44.436 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:44.433+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:44.537 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:44.533+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:44.565 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:44.565 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:44.566 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:44.566 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:44.580 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:44.638 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:44.633+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:44.700 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:44.701 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:44.701 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:44.701 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:44.701 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:44.716 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:44.739 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:44.737+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:58:44.798 INFO:teuthology.orchestra.run.smithi042.stdout:3591 write 0xc88499b thru 0xc891667 (0xcccd bytes) 2024-09-18T00:58:44.817 INFO:teuthology.orchestra.run.smithi042.stdout:3592 read 0x7b98ea5 thru 0x7ba09f8 (0x7b54 bytes) 2024-09-18T00:58:44.818 INFO:teuthology.orchestra.run.smithi042.stdout:3593 punch from 0xbc4df2c to 0xbc5d512, (0xf5e6 bytes) 2024-09-18T00:58:44.819 INFO:teuthology.orchestra.run.smithi042.stdout:3594 write 0x221a437 thru 0x222141e (0x6fe8 bytes) 2024-09-18T00:58:44.840 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:44.833+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:44.895 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1670 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1335296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.727007718999999, 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083399740999999999, 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.896 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1670 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166572+0000", 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166573+0000", 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1829999999999999e-06 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166578+0000", 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0030000000000002e-06 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166594+0000", 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6470999999999999e-05 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166607+0000", 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2831e-05 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249706+0000", 2024-09-18T00:58:44.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083099022999999994 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249843+0000", 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136518 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.249971+0000", 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012871200000000001 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1671 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1376256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726956942000001, 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083700574, 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.901 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1671 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166622+0000", 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166623+0000", 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.066e-06 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166641+0000", 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7728e-05 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166659+0000", 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7850999999999999e-05 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166672+0000", 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3020000000000001e-05 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250034+0000", 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083362085000000002 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250145+0000", 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111045 2024-09-18T00:58:44.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250323+0000", 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017777900000000001 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1672 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726883483000002, 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083853480999999994, 2024-09-18T00:58:44.905 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1672 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166697+0000", 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0920000000000001e-06 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166702+0000", 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0799999999999996e-06 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166718+0000", 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6167999999999998e-05 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166745+0000", 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7177000000000001e-05 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250382+0000", 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083636792000000001 2024-09-18T00:58:44.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250476+0000", 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250549+0000", 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3282999999999996e-05 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1673 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726816374999999, 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084151880999999998, 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1673 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166764+0000", 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166770+0000", 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5969999999999999e-06 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166786+0000", 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6439999999999998e-05 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166799+0000", 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2934e-05 2024-09-18T00:58:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250605+0000", 2024-09-18T00:58:44.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083806357999999997 2024-09-18T00:58:44.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250724+0000", 2024-09-18T00:58:44.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011821500000000001 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250915+0000", 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000191251 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1674 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726716789999998, 2024-09-18T00:58:44.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084239101999999996, 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1674 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166864+0000", 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3629999999999999e-06 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166883+0000", 2024-09-18T00:58:44.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9083e-05 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166902+0000", 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8711000000000001e-05 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166915+0000", 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3375e-05 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250962+0000", 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084047254000000002 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251029+0000", 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6315000000000005e-05 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251102+0000", 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3001000000000001e-05 2024-09-18T00:58:44.918 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1675 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1540096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726644939000003, 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084536891000000003, 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.919 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.920 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1675 2024-09-18T00:58:44.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.920 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:44.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166936+0000", 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2720000000000001e-06 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166941+0000", 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7370000000000003e-06 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166958+0000", 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166972+0000", 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3569e-05 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251159+0000", 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084187467000000002 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251300+0000", 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014059200000000001 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251471+0000", 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017168000000000001 2024-09-18T00:58:44.923 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.924 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.924 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.924 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1676 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.924 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:44.924 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726592058999998, 2024-09-18T00:58:44.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084738637000000006, 2024-09-18T00:58:44.924 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.924 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.924 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1676 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:44.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166988+0000", 2024-09-18T00:58:44.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:58:44.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166993+0000", 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8339999999999998e-06 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167009+0000", 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6028e-05 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167022+0000", 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2675e-05 2024-09-18T00:58:44.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251517+0000", 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084494975999999999 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251576+0000", 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8644999999999998e-05 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251726+0000", 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015039 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1677 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726540387999997, 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085055403000000002, 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.929 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1677 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:44.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167040+0000", 2024-09-18T00:58:44.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:44.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167046+0000", 2024-09-18T00:58:44.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167062+0000", 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6354e-05 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167075+0000", 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2965e-05 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251789+0000", 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084713940000000001 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251916+0000", 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000127176 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252094+0000", 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017833599999999999 2024-09-18T00:58:44.933 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1678 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726488625000002, 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085316746999999998, 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.934 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1678 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167092+0000", 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167097+0000", 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7029999999999997e-06 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167113+0000", 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167125+0000", 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2517000000000001e-05 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252157+0000", 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085031933000000004 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252277+0000", 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119708 2024-09-18T00:58:44.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252407+0000", 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013031800000000001 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1679 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:44.938 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726437906000001, 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085557195000000003, 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1679 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:44.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167143+0000", 2024-09-18T00:58:44.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:58:44.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167170+0000", 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7523000000000001e-05 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167189+0000", 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8530999999999999e-05 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167215+0000", 2024-09-18T00:58:44.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6676999999999999e-05 2024-09-18T00:58:44.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252445+0000", 2024-09-18T00:58:44.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085229416000000002 2024-09-18T00:58:44.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252529+0000", 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4740000000000005e-05 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252699+0000", 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016920800000000001 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1680 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:44.943 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726344449000003, 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085810110999999994, 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1680 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:44.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:44.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167236+0000", 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.048e-06 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167242+0000", 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6459999999999998e-06 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167259+0000", 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.696e-05 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167271+0000", 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.292e-05 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252839+0000", 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085567292000000003 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252924+0000", 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5511000000000002e-05 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253045+0000", 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120734 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.948 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1681 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726238352999999, 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085887343000000005, 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1681 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.949 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:44.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:44.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167342+0000", 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167348+0000", 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4519999999999996e-06 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167372+0000", 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4261999999999999e-05 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167400+0000", 2024-09-18T00:58:44.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-18T00:58:44.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253091+0000", 2024-09-18T00:58:44.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085690895000000003 2024-09-18T00:58:44.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253154+0000", 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3262000000000005e-05 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253228+0000", 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4318000000000002e-05 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.953 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1682 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.954 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:44.954 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.726160272999998, 2024-09-18T00:58:44.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.086061112999999995, 2024-09-18T00:58:44.954 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.954 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.954 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.954 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.954 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.954 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.954 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1682 2024-09-18T00:58:44.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.955 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:44.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:44.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168114+0000", 2024-09-18T00:58:44.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00069532600000000002 2024-09-18T00:58:44.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168123+0000", 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8319999999999995e-06 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168147+0000", 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3536999999999998e-05 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168163+0000", 2024-09-18T00:58:44.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6317e-05 2024-09-18T00:58:44.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253274+0000", 2024-09-18T00:58:44.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085111390999999995 2024-09-18T00:58:44.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253362+0000", 2024-09-18T00:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7384000000000002e-05 2024-09-18T00:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253480+0000", 2024-09-18T00:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011832600000000001 2024-09-18T00:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1683 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1867776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.725383141999998, 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085456638000000001, 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.960 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1683 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:44.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169938+0000", 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017419079999999999 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169951+0000", 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2534999999999999e-05 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169990+0000", 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9004999999999997e-05 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170011+0000", 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.156e-05 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253534+0000", 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083522759000000002 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253585+0000", 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1316999999999997e-05 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253653+0000", 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7553999999999997e-05 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.964 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1684 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.723533506000003, 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083787188999999998, 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1684 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170046+0000", 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170048+0000", 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.674e-06 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170055+0000", 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3549999999999999e-06 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170073+0000", 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7662000000000001e-05 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170087+0000", 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4229e-05 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253701+0000", 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083614346000000006 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253752+0000", 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0760000000000002e-05 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253833+0000", 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1162999999999995e-05 2024-09-18T00:58:44.968 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1685 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.723472708999999, 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083900785000000005, 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.969 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1685 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170108+0000", 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.229e-06 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170114+0000", 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0079999999999999e-06 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170132+0000", 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7734e-05 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170145+0000", 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2977e-05 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253866+0000", 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083721208000000005 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253934+0000", 2024-09-18T00:58:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.834e-05 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254007+0000", 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3288999999999996e-05 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1686 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.973 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.723417498000003, 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084083526000000006, 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1686 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170163+0000", 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1909999999999999e-06 2024-09-18T00:58:44.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170180+0000", 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7323e-05 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170198+0000", 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7723e-05 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170212+0000", 2024-09-18T00:58:44.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3645999999999999e-05 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254050+0000", 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083838115000000005 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254136+0000", 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6106999999999994e-05 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254245+0000", 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010942099999999999 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1687 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.723349196000001, 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084186574, 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.978 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1687 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170647+0000", 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416649 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170655+0000", 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6510000000000007e-06 2024-09-18T00:58:44.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170683+0000", 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7093999999999998e-05 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170701+0000", 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8383e-05 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254283+0000", 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083582091999999997 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254339+0000", 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6187999999999998e-05 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254417+0000", 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7516999999999997e-05 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.982 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1688 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.722853481999998, 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083931026000000006, 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.983 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1688 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173553+0000", 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002827333 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173566+0000", 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3276e-05 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173603+0000", 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6177000000000003e-05 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173628+0000", 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5343000000000001e-05 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254460+0000", 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.080831745999999996 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254547+0000", 2024-09-18T00:58:44.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7045000000000004e-05 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254657+0000", 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011010600000000001 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.118716280000001, 2024-09-18T00:58:44.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:58:44.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:58:44.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:58:44.992 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.993 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.993 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.993 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:44.993 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:44.993 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:44.994 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:44.995 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:44.989+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:45.088 INFO:teuthology.orchestra.run.smithi042.stdout:3595 writesame 0xc0bf196 thru 0xc0c3cef data_size 0xf(0x4b5a bytes) 2024-09-18T00:58:45.094 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:45.089+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:45.103 INFO:teuthology.orchestra.run.smithi042.stdout:3599 punch from 0x9fa99e9 to 0x9fb1b0c, (0x8123 bytes) 2024-09-18T00:58:45.123 INFO:teuthology.orchestra.run.smithi042.stdout:3600 trunc from 0xc891668 to 0x203f455 2024-09-18T00:58:45.195 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:45.189+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:45.197 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:45.198 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:45.198 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:45.198 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:45.218 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:45.305 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:45.301+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:45.344 INFO:teuthology.orchestra.run.smithi042.stdout:3601 trunc from 0x203f455 to 0x338f655 2024-09-18T00:58:45.354 INFO:teuthology.orchestra.run.smithi042.stdout:3602 read 0x111bc7f thru 0x111c58c (0x90e bytes) 2024-09-18T00:58:45.356 INFO:teuthology.orchestra.run.smithi042.stdout:3603 read 0x21f2754 thru 0x21f5cd8 (0x3585 bytes) 2024-09-18T00:58:45.384 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:45.385 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:45.385 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:45.385 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:45.385 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:45.397 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:45.393+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:45.399 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:45.409 INFO:teuthology.orchestra.run.smithi042.stdout:3604 writesame 0x4b0ddbd thru 0x4b12a77 data_size 0x1(0x4cbb bytes) 2024-09-18T00:58:45.434 INFO:teuthology.orchestra.run.smithi042.stdout:3605 write 0xaa21ec5 thru 0xaa257ec (0x3928 bytes) 2024-09-18T00:58:45.499 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:45.493+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:45.522 INFO:teuthology.orchestra.run.smithi042.stdout:3606 read 0x5de6126 thru 0x5def824 (0x96ff bytes) 2024-09-18T00:58:45.522 INFO:teuthology.orchestra.run.smithi042.stdout:3607 read 0x7a329dd thru 0x7a3f034 (0xc658 bytes) 2024-09-18T00:58:45.522 INFO:teuthology.orchestra.run.smithi042.stdout:3608 clone 29 order 19 su 4096 sc 14 2024-09-18T00:58:45.557 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.145511956, 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:45.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T00:58:45.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T00:58:45.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:26797 3.0 3:03a9420d:::rbd_data.117c4c7db1a0.0000000000000071:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.367085156999998, 2024-09-18T00:58:45.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073789632999999993, 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 26797 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187640+0000", 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187653+0000", 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2357e-05 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187684+0000", 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1582999999999999e-05 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187699+0000", 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5003e-05 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187806+0000", 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106776 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.188026+0000", 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021975799999999999 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7", 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.201663+0000", 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013637768 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261226+0000", 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059562916 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261261+0000", 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4360000000000003e-05 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261354+0000", 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3041999999999995e-05 2024-09-18T00:58:45.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261427+0000", 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3553000000000005e-05 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64346 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965833571000001, 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048905196999999997, 2024-09-18T00:58:45.570 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64346 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588890+0000", 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3e-07 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588893+0000", 2024-09-18T00:58:45.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.106e-06 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588904+0000", 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1037e-05 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588911+0000", 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2729999999999997e-06 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637426+0000", 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048514483999999997 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637537+0000", 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110901 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637795+0000", 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025776599999999998 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.574 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64347 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965801698, 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049469502999999998, 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.575 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64347 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588922+0000", 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4300000000000003e-07 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588925+0000", 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588934+0000", 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0329999999999997e-06 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588946+0000", 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1547e-05 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637872+0000", 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048926202000000002 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638047+0000", 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174566 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638391+0000", 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034401499999999998 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.579 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64348 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965760828000001, 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050239487999999999, 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64348 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588963+0000", 2024-09-18T00:58:45.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.133e-06 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588969+0000", 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4229999999999996e-06 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588996+0000", 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7396e-05 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589163+0000", 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016677900000000001 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638498+0000", 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049334743 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638715+0000", 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000217414 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639202+0000", 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048660000000000001 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64349 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965531923, 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050607656000000001, 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:45.584 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64349 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589192+0000", 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.536e-06 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589201+0000", 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-06 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589222+0000", 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-18T00:58:45.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589241+0000", 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8960000000000001e-05 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639285+0000", 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050043892999999999 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639457+0000", 2024-09-18T00:58:45.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017286299999999999 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639799+0000", 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034117400000000002 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64350 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965461577999999, 2024-09-18T00:58:45.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051028502000000003, 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64350 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589262+0000", 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1570000000000001e-06 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589280+0000", 2024-09-18T00:58:45.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7943000000000001e-05 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589299+0000", 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8940000000000002e-05 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589316+0000", 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6654e-05 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639872+0000", 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050556151000000001 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639988+0000", 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115423 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640290+0000", 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030223400000000002 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64351 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965385838, 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051389975999999997, 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64351 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589338+0000", 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589345+0000", 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589363+0000", 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8346e-05 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589381+0000", 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7357e-05 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640373+0000", 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050992347 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640504+0000", 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000131011 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640727+0000", 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022310799999999999 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64352 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965322309000001, 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051932649999999997, 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.597 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64352 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589402+0000", 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.237e-06 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589408+0000", 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2770000000000002e-06 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589426+0000", 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8006e-05 2024-09-18T00:58:45.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589441+0000", 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4934e-05 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640803+0000", 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051361619999999997 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640986+0000", 2024-09-18T00:58:45.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018303199999999999 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641333+0000", 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034754399999999998 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64353 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:45.601 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965263915000001, 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052565148999999999, 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64353 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589460+0000", 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4300000000000001e-06 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589466+0000", 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.446e-06 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589485+0000", 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8726999999999999e-05 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589499+0000", 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.402e-05 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641427+0000", 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051927945000000003 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641601+0000", 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174859 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642024+0000", 2024-09-18T00:58:45.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000422722 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64354 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965189215999999, 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053055607999999997, 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:45.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64354 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:45.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589535+0000", 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3430000000000001e-06 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589542+0000", 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589561+0000", 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9207000000000001e-05 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589577+0000", 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5933000000000001e-05 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642111+0000", 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052533799999999999 2024-09-18T00:58:45.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642271+0000", 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015998499999999999 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642589+0000", 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031818999999999999 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64355 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965125180000001, 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053570327000000001, 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64355 2024-09-18T00:58:45.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589599+0000", 2024-09-18T00:58:45.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589605+0000", 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8830000000000003e-06 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589622+0000", 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.732e-05 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589636+0000", 2024-09-18T00:58:45.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3703000000000001e-05 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642672+0000", 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053036204000000003 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642828+0000", 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155451 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643168+0000", 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034051900000000001 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64356 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965067667, 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054203299000000003, 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.615 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64356 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589656+0000", 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589662+0000", 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9800000000000003e-06 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589681+0000", 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8300000000000001e-05 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589695+0000", 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3997e-05 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643249+0000", 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053554101999999999 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643423+0000", 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174062 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643859+0000", 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043562499999999998 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64357 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:45.619 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.965009717000001, 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054239849, 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64357 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589714+0000", 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.266e-06 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589720+0000", 2024-09-18T00:58:45.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2340000000000002e-06 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589738+0000", 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7932999999999999e-05 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589752+0000", 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4867e-05 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643934+0000", 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054181721000000002 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643953+0000", 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8828e-05 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64358 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.623 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.964951838000001, 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054261125, 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64358 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589772+0000", 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1170000000000001e-06 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589778+0000", 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8429999999999998e-06 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589799+0000", 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0727e-05 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589813+0000", 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4403000000000001e-05 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644021+0000", 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054207784000000002 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644032+0000", 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1251e-05 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64359 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.627 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.964889872000001, 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286586999999997, 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64359 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589834+0000", 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589841+0000", 2024-09-18T00:58:45.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.956e-06 2024-09-18T00:58:45.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589864+0000", 2024-09-18T00:58:45.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3124000000000001e-05 2024-09-18T00:58:45.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589879+0000", 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.486e-05 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644099+0000", 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054219429 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644120+0000", 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1004000000000001e-05 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64360 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.964825678, 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054290699999999997, 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.633 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64360 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589898+0000", 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3600000000000002e-07 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589903+0000", 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4669999999999998e-06 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589913+0000", 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0463e-05 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589921+0000", 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7959999999999996e-06 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644173+0000", 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054251903999999997 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644188+0000", 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5133999999999999e-05 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64361 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.964792199000001, 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054320101000000003, 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.637 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64361 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5000000000000002e-07 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589935+0000", 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2799999999999999e-06 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589944+0000", 2024-09-18T00:58:45.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589954+0000", 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0036e-05 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644240+0000", 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286273000000003 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644251+0000", 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0392e-05 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64362 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.964759053000002, 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054340196, 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:45.641 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64362 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:45.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589978+0000", 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3570000000000001e-05 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589991+0000", 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3196999999999999e-05 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590000+0000", 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2800000000000003e-06 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644294+0000", 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054294873 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644304+0000", 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6560000000000004e-06 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64363 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.964713023000002, 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054356660000000001, 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:45.645 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64363 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590011+0000", 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7400000000000003e-07 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590014+0000", 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590024+0000", 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5670000000000001e-06 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590031+0000", 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3830000000000003e-06 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644354+0000", 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054323429999999999 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644367+0000", 2024-09-18T00:58:45.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.217e-05 2024-09-18T00:58:45.649 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.649 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.649 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.649 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:45.649 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:45.649 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:45.650 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:45.651 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:45.645+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:45.752 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:45.749+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:45.784 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:45.785 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:45.785 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:45.785 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:45.800 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:45.849 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:45.852 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:45.845+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:45.917 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:45.917 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:45.917 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:45.917 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:45.917 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:45.932 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:45.974 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:45.969+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:46.010 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:46.011 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:46.026 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:46.027 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:46.043 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:46.044 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:46.054 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:46.049+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:46.062 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:46.062 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:46.082 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:46.082 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:46.101 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:46.101 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:46.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 57.009604219000003, 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:46.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T00:58:46.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.618709268000003, 2024-09-18T00:58:46.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T00:58:46.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.110 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78503 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2699851469999999, 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053463986999999998, 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78503 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6899999999999997e-07 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828236+0000", 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.853e-06 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828243+0000", 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4359999999999996e-06 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828257+0000", 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3862999999999999e-05 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881475+0000", 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053217299000000003 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881569+0000", 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4744e-05 2024-09-18T00:58:46.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881697+0000", 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012722300000000001 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78504 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 897024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.269951243, 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053679381999999998, 2024-09-18T00:58:46.115 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78504 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828572+0000", 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030532900000000002 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828577+0000", 2024-09-18T00:58:46.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8579999999999999e-06 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828596+0000", 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9414000000000001e-05 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828604+0000", 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0190000000000006e-06 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881742+0000", 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053137864 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881822+0000", 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0346000000000006e-05 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881946+0000", 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123552 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.119 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78505 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.269597555, 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053688825000000003, 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78505 2024-09-18T00:58:46.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828656+0000", 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6273999999999999e-05 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828660+0000", 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.76e-06 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828670+0000", 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5279999999999992e-06 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828678+0000", 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8560000000000007e-06 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881994+0000", 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053316364999999998 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882203+0000", 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020851600000000001 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882309+0000", 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106526 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78506 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2695287450000001, 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032405999999998, 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.124 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78506 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828976+0000", 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028726700000000001 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828983+0000", 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.246e-06 2024-09-18T00:58:46.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828996+0000", 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3125999999999999e-05 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829003+0000", 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6899999999999992e-06 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882345+0000", 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053341462999999999 2024-09-18T00:58:46.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882423+0000", 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8338999999999997e-05 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882721+0000", 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029827500000000003 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78507 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2692014839999999, 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053936698999999998, 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78507 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829065+0000", 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8863999999999998e-05 2024-09-18T00:58:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829069+0000", 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9160000000000003e-06 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829080+0000", 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1087000000000001e-05 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829088+0000", 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2609999999999993e-06 2024-09-18T00:58:46.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882766+0000", 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053678037999999997 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882847+0000", 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0131999999999998e-05 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882953+0000", 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106401 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78508 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 978944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2691180659999999, 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053997636000000002, 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.133 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78508 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830739+0000", 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001639738 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830745+0000", 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8869999999999997e-06 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830755+0000", 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8830000000000001e-06 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830765+0000", 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7589999999999992e-06 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882990+0000", 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052225252 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.883040+0000", 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9848000000000001e-05 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.883097+0000", 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7268999999999997e-05 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78514 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1101824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2673104369999999, 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053776592999999998, 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78514 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830908+0000", 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830928+0000", 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9898000000000002e-05 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830940+0000", 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2483999999999999e-05 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830955+0000", 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5143999999999999e-05 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884176+0000", 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053221151000000001 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884581+0000", 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040494199999999999 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884684+0000", 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102476 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78515 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1122304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2672527220000001, 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053953972000000003, 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78515 2024-09-18T00:58:46.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830966+0000", 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4799999999999998e-07 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830968+0000", 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6390000000000001e-06 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830975+0000", 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4790000000000001e-06 2024-09-18T00:58:46.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830980+0000", 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4269999999999999e-06 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884731+0000", 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053750345999999997 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884785+0000", 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.49e-05 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884919+0000", 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133533 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78516 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.267229154, 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054090795999999997, 2024-09-18T00:58:46.146 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78516 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830991+0000", 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.221e-06 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830998+0000", 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5409999999999997e-06 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831003+0000", 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2390000000000002e-06 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884953+0000", 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053950117999999998 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885000+0000", 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6555999999999999e-05 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885079+0000", 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9702000000000004e-05 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.150 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78517 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1163264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2672082179999999, 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298990999999998, 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78517 2024-09-18T00:58:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4400000000000001e-07 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831018+0000", 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2490000000000001e-06 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831025+0000", 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831031+0000", 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.463e-06 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885130+0000", 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054099371 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885207+0000", 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7292000000000005e-05 2024-09-18T00:58:46.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885309+0000", 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010129899999999999 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78518 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1183744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.267178439, 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054532499999999998, 2024-09-18T00:58:46.155 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78518 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831040+0000", 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3099999999999997e-07 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831042+0000", 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2879999999999999e-06 2024-09-18T00:58:46.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831050+0000", 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.464e-06 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831055+0000", 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0819999999999998e-06 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885354+0000", 2024-09-18T00:58:46.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298739999999998 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885428+0000", 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4567999999999995e-05 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885572+0000", 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014372699999999999 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78519 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1204224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2671565899999999, 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054707261, 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78519 2024-09-18T00:58:46.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831062+0000", 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831064+0000", 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0360000000000001e-06 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831069+0000", 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4149999999999998e-06 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831075+0000", 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2249999999999997e-06 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885602+0000", 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054526893 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885673+0000", 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0865000000000007e-05 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885768+0000", 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5422999999999998e-05 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78520 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1224704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.267136134, 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054906695999999998, 2024-09-18T00:58:46.164 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78520 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831084+0000", 2024-09-18T00:58:46.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2960000000000001e-06 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831091+0000", 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831095+0000", 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.792e-06 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885802+0000", 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054706795000000003 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885884+0000", 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2297000000000002e-05 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885988+0000", 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010393199999999999 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.168 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78521 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.267091261, 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055066616999999998, 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.169 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78521 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831127+0000", 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.99e-07 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831130+0000", 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6819999999999999e-06 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831137+0000", 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3849999999999996e-06 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831145+0000", 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7360000000000001e-06 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886020+0000", 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054874700999999998 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886096+0000", 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6234000000000002e-05 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886193+0000", 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7280000000000004e-05 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78522 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2670655, 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055137938999999997, 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.173 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78522 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831153+0000", 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.37e-07 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831155+0000", 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4420000000000001e-06 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831161+0000", 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831166+0000", 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8749999999999999e-06 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886279+0000", 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055112337999999997 2024-09-18T00:58:46.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886290+0000", 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1527e-05 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78523 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.26704482, 2024-09-18T00:58:46.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055171425000000003, 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78523 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0900000000000002e-07 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831176+0000", 2024-09-18T00:58:46.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4830000000000002e-06 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831183+0000", 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0679999999999999e-06 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831189+0000", 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886331+0000", 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055142669999999998 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886344+0000", 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2918999999999999e-05 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78524 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.181 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2670227039999999, 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055194430000000003, 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78524 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831196+0000", 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1200000000000003e-07 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831198+0000", 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831204+0000", 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2740000000000004e-06 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831209+0000", 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0900000000000004e-06 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886381+0000", 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055172142 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886389+0000", 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2150000000000001e-06 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78525 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:46.185 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.267002191, 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055206535000000001, 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78525 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:46.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832358+0000", 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001142531 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:46.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832364+0000", 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832375+0000", 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1381e-05 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832382+0000", 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9800000000000001e-06 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886415+0000", 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032678000000001 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886422+0000", 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4449999999999999e-06 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:46.189 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:46.191 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:46.192 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:46.192 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:46.193 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:46.193 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:46.185+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:46.292 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:46.285+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:46.360 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:46.360 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:46.375 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:46.393 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:46.389+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:46.494 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:46.493+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:46.541 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:46.541 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:46.555 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:46.601 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:46.597+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:46.677 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:46.678 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:46.691 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:46.738 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:46.733+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:46.796 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:46.789+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:46.809 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:58:46.858 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:46.858 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:46.877 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:46.894 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone28 from 0xaa257ed (overlap 0x203f455) to 0x9ac94c 2024-09-18T00:58:46.897 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:46.893+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:46.998 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:46.993+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:47.026 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:47.026 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:47.044 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:47.099 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:47.093+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:47.200 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:47.193+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:47.232 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:47.233 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:47.249 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:47.301 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:47.297+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:47.402 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:47.397+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:47.429 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:47.429 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:47.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:47.502 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:47.497+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:47.574 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:47.574 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:47.587 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:47.630 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:47.625+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:47.704 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:47.697+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:47.747 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:47.747 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:47.761 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:47.786 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:58:47.786 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:58:47.786 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-18T00:58:47.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-18T00:58:47.789 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:58:47.812 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:47.805+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:47.906 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:47.901+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:47.911 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:47.911 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:47.928 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:48.007 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:48.001+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:48.087 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:48.087 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:48.100 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:48.102 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-18T00:58:48.093+0000 7fdb35eeb640 -1 mon.a@0(leader).osd e78 definitely_dead 0 2024-09-18T00:58:48.150 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:48.145+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:48.195 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:48.210 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:48.211 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:48.211 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:48.205+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:48.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:48.310 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:48.305+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:48.337 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:48.337 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:48.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:48.371 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:48.387 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:48.387 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:48.404 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:48.404 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:48.410 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:48.405+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:48.419 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:48.419 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:48.419 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:58:48.419 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:48.423 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:48.424 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:48.432 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:48.444 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:48.444 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:48.464 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:48.465 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:48.484 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:48.484 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:48.542 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:48.537+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:58:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:58:48.571 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:58:48.571 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:48.588 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:48.612 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:48.605+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:48.712 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:48.705+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:48.757 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.757 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 81.441365359000002, 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101264911, 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4226", 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2155402750", 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313760+0000", 2024-09-18T00:58:48.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.612e-06 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313768+0000", 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9249999999999995e-06 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313786+0000", 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8253999999999998e-05 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313803+0000", 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313935+0000", 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013180399999999999 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313954+0000", 2024-09-18T00:58:48.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8768999999999999e-05 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.411874+0000", 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.097920646 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412292+0000", 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041777699999999998 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412675+0000", 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038262300000000001 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414019+0000", 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013436699999999999 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414872+0000", 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085320999999999997 2024-09-18T00:58:48.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414955+0000", 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3405999999999994e-05 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.415023+0000", 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7825999999999997e-05 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15500 3.11 3:8ed6a97b:::rbd_data.117c53e529c1.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.940605339000001, 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046818956000000002, 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15500 2024-09-18T00:58:48.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814521+0000", 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.977e-06 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814536+0000", 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.487e-05 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814572+0000", 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5567e-05 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814588+0000", 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6546e-05 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814675+0000", 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6675999999999995e-05 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814839+0000", 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016414199999999999 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.819908+0000", 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050695280000000002 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861139+0000", 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041230771999999999 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861208+0000", 2024-09-18T00:58:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8722999999999999e-05 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861277+0000", 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9246999999999996e-05 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861337+0000", 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9908000000000003e-05 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87779 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1404928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.534897+0000", 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.220226752, 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045551541000000001, 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87779 2024-09-18T00:58:48.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534897+0000", 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534897+0000", 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534897+0000", 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534898+0000", 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0759999999999999e-06 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534901+0000", 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1810000000000001e-06 2024-09-18T00:58:48.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534904+0000", 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6620000000000001e-06 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580264+0000", 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045360294000000002 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580350+0000", 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5556000000000006e-05 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580448+0000", 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8548999999999994e-05 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87780 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.534907+0000", 2024-09-18T00:58:48.775 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.220216408, 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045657725000000003, 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87780 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534907+0000", 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534907+0000", 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534907+0000", 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.41e-07 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534908+0000", 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534912+0000", 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.681e-06 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534916+0000", 2024-09-18T00:58:48.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.442e-06 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580463+0000", 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045546924000000003 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580510+0000", 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6390999999999997e-05 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580565+0000", 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4968e-05 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87781 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1445888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.534943+0000", 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.22018063700000001, 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045839058000000002, 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87781 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534943+0000", 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534943+0000", 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534943+0000", 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2399999999999999e-07 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534946+0000", 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4980000000000001e-06 2024-09-18T00:58:48.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534953+0000", 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.985e-06 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534958+0000", 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5860000000000004e-06 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580602+0000", 2024-09-18T00:58:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045643209999999997 2024-09-18T00:58:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580684+0000", 2024-09-18T00:58:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2539000000000004e-05 2024-09-18T00:58:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580782+0000", 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7816e-05 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87782 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1466368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.534966+0000", 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.22015726399999999, 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045923248, 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87782 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534966+0000", 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534966+0000", 2024-09-18T00:58:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534967+0000", 2024-09-18T00:58:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.58e-07 2024-09-18T00:58:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534969+0000", 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1940000000000001e-06 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534977+0000", 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7679999999999997e-06 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534983+0000", 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7180000000000001e-06 2024-09-18T00:58:48.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580798+0000", 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045814663999999998 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580837+0000", 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9316000000000001e-05 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580889+0000", 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.223e-05 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87783 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1486848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.534990+0000", 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.22013295999999999, 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046128573999999999, 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87783 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534990+0000", 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534990+0000", 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534991+0000", 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534993+0000", 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:58:48.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535004+0000", 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0983e-05 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535012+0000", 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0199999999999994e-06 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580928+0000", 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045915456 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581007+0000", 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9556999999999997e-05 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581119+0000", 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111864 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.794 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87784 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1507328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535020+0000", 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.22010301400000001, 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046227183999999998, 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87784 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535020+0000", 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535020+0000", 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535021+0000", 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535023+0000", 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2929999999999999e-06 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535030+0000", 2024-09-18T00:58:48.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9859999999999997e-06 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535035+0000", 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1499999999999998e-06 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581153+0000", 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046117461999999998 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581195+0000", 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2076999999999997e-05 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581248+0000", 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-05 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.799 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87785 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1527808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535042+0000", 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.220081474, 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046410976, 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87785 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535042+0000", 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535042+0000", 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535042+0000", 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9499999999999998e-07 2024-09-18T00:58:48.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535044+0000", 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0940000000000002e-06 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535052+0000", 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0229999999999992e-06 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535061+0000", 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9509999999999995e-06 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581282+0000", 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046221048000000001 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581357+0000", 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4912000000000004e-05 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581453+0000", 2024-09-18T00:58:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5552999999999998e-05 2024-09-18T00:58:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87786 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1548288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535069+0000", 2024-09-18T00:58:48.804 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.220054052, 2024-09-18T00:58:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046523549999999997, 2024-09-18T00:58:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87786 2024-09-18T00:58:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535069+0000", 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535069+0000", 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535070+0000", 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.96e-07 2024-09-18T00:58:48.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535097+0000", 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7526999999999999e-05 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535106+0000", 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2339999999999994e-06 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535112+0000", 2024-09-18T00:58:48.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581490+0000", 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04637819 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581534+0000", 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4283000000000003e-05 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581593+0000", 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8888999999999998e-05 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87787 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535124+0000", 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.21999988400000001, 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046685899000000003, 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87787 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535124+0000", 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535124+0000", 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535124+0000", 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8100000000000003e-07 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535127+0000", 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5459999999999998e-06 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535135+0000", 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4020000000000002e-06 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535141+0000", 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6559999999999997e-06 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581633+0000", 2024-09-18T00:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046492646999999998 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581717+0000", 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3815999999999998e-05 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581809+0000", 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2250999999999995e-05 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87788 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1589248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535148+0000", 2024-09-18T00:58:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.21997501999999999, 2024-09-18T00:58:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046785392000000002, 2024-09-18T00:58:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.814 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87788 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535148+0000", 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535148+0000", 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535277+0000", 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000129071 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535280+0000", 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9000000000000002e-06 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535287+0000", 2024-09-18T00:58:48.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.618e-06 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535291+0000", 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2250000000000002e-06 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581841+0000", 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046549556999999998 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581882+0000", 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1273000000000003e-05 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581934+0000", 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1748e-05 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87789 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1609728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.818 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535308+0000", 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.219815921, 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046832960999999999, 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87789 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535308+0000", 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535308+0000", 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535418+0000", 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110726 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535421+0000", 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4609999999999998e-06 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535428+0000", 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535435+0000", 2024-09-18T00:58:48.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5089999999999999e-06 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581970+0000", 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046535853000000002 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582047+0000", 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6457000000000003e-05 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582140+0000", 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3597999999999994e-05 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87790 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1630208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535445+0000", 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.21967831099999999, 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046815748999999997, 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87790 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535445+0000", 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535445+0000", 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535640+0000", 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019530599999999999 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535644+0000", 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6320000000000001e-06 2024-09-18T00:58:48.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535657+0000", 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2957e-05 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535662+0000", 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4970000000000003e-06 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582171+0000", 2024-09-18T00:58:48.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046509485000000003 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582212+0000", 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0844999999999999e-05 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582261+0000", 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9026999999999997e-05 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.827 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87791 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535673+0000", 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.219450598, 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046637830999999998, 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87791 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535673+0000", 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535673+0000", 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536311+0000", 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063850300000000001 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536316+0000", 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.408e-06 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536325+0000", 2024-09-18T00:58:48.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7380000000000006e-06 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536338+0000", 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2099000000000001e-05 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582299+0000", 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045961850999999998 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582311+0000", 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1232e-05 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87792 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.536351+0000", 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.21877260900000001, 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045989887, 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87792 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536351+0000", 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536351+0000", 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536351+0000", 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536354+0000", 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6410000000000002e-06 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536361+0000", 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4229999999999999e-06 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536368+0000", 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.72e-06 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582334+0000", 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045966295999999997 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582341+0000", 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3029999999999996e-06 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87793 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.536393+0000", 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.21873051399999999, 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045987987000000001, 2024-09-18T00:58:48.836 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87793 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536393+0000", 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536393+0000", 2024-09-18T00:58:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536393+0000", 2024-09-18T00:58:48.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9399999999999995e-07 2024-09-18T00:58:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536396+0000", 2024-09-18T00:58:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:58:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536405+0000", 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8820000000000007e-06 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536411+0000", 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1619999999999996e-06 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582374+0000", 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045962692999999999 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582381+0000", 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8399999999999997e-06 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87794 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.536421+0000", 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.218702649, 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045991835000000002, 2024-09-18T00:58:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87794 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536421+0000", 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536421+0000", 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536421+0000", 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536424+0000", 2024-09-18T00:58:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7659999999999999e-06 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536429+0000", 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7369999999999997e-06 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536432+0000", 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1520000000000001e-06 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582406+0000", 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045974358 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582413+0000", 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000002e-06 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87795 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.536446+0000", 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.218677331, 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045992551999999999, 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87795 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536446+0000", 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536446+0000", 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536664+0000", 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000217636 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536668+0000", 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9269999999999998e-06 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536676+0000", 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0960000000000001e-06 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536683+0000", 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7569999999999998e-06 2024-09-18T00:58:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582433+0000", 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045750199999999998 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582439+0000", 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9359999999999996e-06 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87796 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.536693+0000", 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.218430812, 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045774065000000003, 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87796 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536693+0000", 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536693+0000", 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.537106+0000", 2024-09-18T00:58:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000413075 2024-09-18T00:58:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.537109+0000", 2024-09-18T00:58:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-18T00:58:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.537117+0000", 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8909999999999995e-06 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.537121+0000", 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5249999999999999e-06 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582460+0000", 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045338090999999997 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582467+0000", 2024-09-18T00:58:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1080000000000004e-06 2024-09-18T00:58:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:48.854 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:48.856 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:48.856 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-18T00:58:48.857 INFO:teuthology.orchestra.run.smithi042.stderr:marked in osd.2. 2024-09-18T00:58:48.857 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-18T00:58:48.857 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:48.849+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:48.957 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:48.953+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:48.988 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:88808 3.2 3.6fe90502 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.847055+0000", 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.141683751, 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.14172696100000001, 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "waiting for peered", 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 88808 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.847055+0000", 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.847055+0000", 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.847057+0000", 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.407e-06 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.847063+0000", 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8229999999999997e-06 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.847077+0000", 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3804e-05 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.847086+0000", 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4030000000000007e-06 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.847242+0000", 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015604900000000001 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.847272+0000", 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0775999999999998e-05 2024-09-18T00:58:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:48.993 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:48.993 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:48.993 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:48.993 INFO:teuthology.orchestra.run.smithi195.stdout: ], 2024-09-18T00:58:48.993 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 1 2024-09-18T00:58:48.993 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:49.002 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:49.089 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:49.085+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:49.159 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:49.153+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:49.166 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:49.166 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:49.167 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:49.167 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:49.167 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:49.181 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:49.269 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:49.265+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:49.345 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:49.345 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:49.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:49.345 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:49.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.345 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1672 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.345 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:49.345 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.175034483000005, 2024-09-18T00:58:49.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083853480999999994, 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1672 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166697+0000", 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0920000000000001e-06 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166702+0000", 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0799999999999996e-06 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166718+0000", 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6167999999999998e-05 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166745+0000", 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7177000000000001e-05 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250382+0000", 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083636792000000001 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250476+0000", 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250549+0000", 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3282999999999996e-05 2024-09-18T00:58:49.349 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1673 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.174967374999994, 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084151880999999998, 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1673 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166764+0000", 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166770+0000", 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5969999999999999e-06 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166786+0000", 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6439999999999998e-05 2024-09-18T00:58:49.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166799+0000", 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2934e-05 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250605+0000", 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083806357999999997 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250724+0000", 2024-09-18T00:58:49.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011821500000000001 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250915+0000", 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000191251 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1674 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:49.354 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.174867789999993, 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084239101999999996, 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1674 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:49.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166864+0000", 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3629999999999999e-06 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166883+0000", 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9083e-05 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166902+0000", 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8711000000000001e-05 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166915+0000", 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3375e-05 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250962+0000", 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084047254000000002 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251029+0000", 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6315000000000005e-05 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251102+0000", 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3001000000000001e-05 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.358 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1675 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1540096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.174795939000006, 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084536891000000003, 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.359 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1675 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166936+0000", 2024-09-18T00:58:49.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2720000000000001e-06 2024-09-18T00:58:49.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166941+0000", 2024-09-18T00:58:49.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7370000000000003e-06 2024-09-18T00:58:49.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166958+0000", 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166972+0000", 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3569e-05 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251159+0000", 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084187467000000002 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251300+0000", 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014059200000000001 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251471+0000", 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017168000000000001 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1676 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.174743058999994, 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084738637000000006, 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1676 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.364 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:49.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166988+0000", 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166993+0000", 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8339999999999998e-06 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167009+0000", 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6028e-05 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167022+0000", 2024-09-18T00:58:49.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2675e-05 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251517+0000", 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084494975999999999 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251576+0000", 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8644999999999998e-05 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251726+0000", 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015039 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1677 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.174691387999999, 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085055403000000002, 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.369 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1677 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167040+0000", 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167046+0000", 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167062+0000", 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6354e-05 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167075+0000", 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2965e-05 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251789+0000", 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084713940000000001 2024-09-18T00:58:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251916+0000", 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000127176 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252094+0000", 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017833599999999999 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1678 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.174639624999998, 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085316746999999998, 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1678 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.374 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167092+0000", 2024-09-18T00:58:49.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167097+0000", 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7029999999999997e-06 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167113+0000", 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167125+0000", 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2517000000000001e-05 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252157+0000", 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085031933000000004 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252277+0000", 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119708 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252407+0000", 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013031800000000001 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1679 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.174588905999997, 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085557195000000003, 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.378 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1679 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167143+0000", 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167170+0000", 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7523000000000001e-05 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167189+0000", 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8530999999999999e-05 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167215+0000", 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6676999999999999e-05 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252445+0000", 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085229416000000002 2024-09-18T00:58:49.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252529+0000", 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4740000000000005e-05 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252699+0000", 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016920800000000001 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1680 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.174495449000005, 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085810110999999994, 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1680 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.383 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167236+0000", 2024-09-18T00:58:49.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.048e-06 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167242+0000", 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6459999999999998e-06 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167259+0000", 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.696e-05 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167271+0000", 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.292e-05 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252839+0000", 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085567292000000003 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252924+0000", 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5511000000000002e-05 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253045+0000", 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120734 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1681 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.174389352999995, 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085887343000000005, 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1681 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167342+0000", 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167348+0000", 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4519999999999996e-06 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167372+0000", 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4261999999999999e-05 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167400+0000", 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253091+0000", 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085690895000000003 2024-09-18T00:58:49.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253154+0000", 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3262000000000005e-05 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253228+0000", 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4318000000000002e-05 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.391 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1682 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.174311273000001, 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.086061112999999995, 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1682 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168114+0000", 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00069532600000000002 2024-09-18T00:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168123+0000", 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8319999999999995e-06 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168147+0000", 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3536999999999998e-05 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168163+0000", 2024-09-18T00:58:49.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6317e-05 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253274+0000", 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085111390999999995 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253362+0000", 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7384000000000002e-05 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253480+0000", 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011832600000000001 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1683 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1867776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.173534141999994, 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085456638000000001, 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.396 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1683 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:49.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169938+0000", 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017419079999999999 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169951+0000", 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2534999999999999e-05 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169990+0000", 2024-09-18T00:58:49.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9004999999999997e-05 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170011+0000", 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.156e-05 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253534+0000", 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083522759000000002 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253585+0000", 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1316999999999997e-05 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253653+0000", 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7553999999999997e-05 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1685 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:49.400 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.171623709000002, 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083900785000000005, 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1685 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:49.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170108+0000", 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.229e-06 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170114+0000", 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0079999999999999e-06 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170132+0000", 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7734e-05 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170145+0000", 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2977e-05 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253866+0000", 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083721208000000005 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253934+0000", 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.834e-05 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254007+0000", 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3288999999999996e-05 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.404 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1686 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.171568497999999, 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084083526000000006, 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.405 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1686 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170163+0000", 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1909999999999999e-06 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170180+0000", 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7323e-05 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170198+0000", 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7723e-05 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170212+0000", 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3645999999999999e-05 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254050+0000", 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083838115000000005 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254136+0000", 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6106999999999994e-05 2024-09-18T00:58:49.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254245+0000", 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010942099999999999 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1687 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.171500195999997, 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084186574, 2024-09-18T00:58:49.409 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1687 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170647+0000", 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416649 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170655+0000", 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6510000000000007e-06 2024-09-18T00:58:49.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170683+0000", 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7093999999999998e-05 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170701+0000", 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8383e-05 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254283+0000", 2024-09-18T00:58:49.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083582091999999997 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254339+0000", 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6187999999999998e-05 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254417+0000", 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7516999999999997e-05 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1688 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.171004482000001, 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083931026000000006, 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1688 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173553+0000", 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002827333 2024-09-18T00:58:49.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173566+0000", 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3276e-05 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173603+0000", 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6177000000000003e-05 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173628+0000", 2024-09-18T00:58:49.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5343000000000001e-05 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254460+0000", 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.080831745999999996 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254547+0000", 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7045000000000004e-05 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254657+0000", 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011010600000000001 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.566867279999997, 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.418 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:49.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:58:49.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:58:49.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:58:49.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:58:49.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87406 3.19 3:9d734bda:::rbd_data.117c8934add3.000000000000007c:head [stat,delete] snapc 1f=[1f] RETRY=8 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.91894415799999996, 2024-09-18T00:58:49.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097024104999999999, 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87406 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425532+0000", 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027458309999999998 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425537+0000", 2024-09-18T00:58:49.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2839999999999997e-06 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425551+0000", 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3682999999999999e-05 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425559+0000", 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5329999999999997e-06 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515035+0000", 2024-09-18T00:58:49.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.089476411000000006 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515168+0000", 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013309699999999999 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515475+0000", 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030662199999999999 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519553+0000", 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040781259999999996 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519665+0000", 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112669 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519748+0000", 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2270999999999999e-05 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519810+0000", 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2577999999999996e-05 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87408 3.19 3:9f063cdd:::rbd_data.117c8934add3.000000000000007d:head [stat,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425586+0000", 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.91614405499999996, 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.094380550999999993, 2024-09-18T00:58:49.430 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87408 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425586+0000", 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425586+0000", 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425587+0000", 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8599999999999998e-07 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425590+0000", 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6790000000000001e-06 2024-09-18T00:58:49.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425596+0000", 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3690000000000003e-06 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425600+0000", 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5549999999999996e-06 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.516522+0000", 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.090921960999999996 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.516767+0000", 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000244202 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.517317+0000", 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055067199999999999 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519642+0000", 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023243790000000001 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519827+0000", 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184816 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519909+0000", 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2799000000000005e-05 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519967+0000", 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7533e-05 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87414 3.19 3:9c145b25:::rbd_data.117c8934add3.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.91612061899999997, 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095886693999999995, 2024-09-18T00:58:49.436 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87414 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425618+0000", 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9810000000000003e-06 2024-09-18T00:58:49.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425625+0000", 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0870000000000001e-06 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425630+0000", 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.262e-06 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518417+0000", 2024-09-18T00:58:49.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092787816999999995 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518593+0000", 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017569100000000001 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518939+0000", 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034572299999999998 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:49.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521320+0000", 2024-09-18T00:58:49.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023809619999999999 2024-09-18T00:58:49.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:49.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521343+0000", 2024-09-18T00:58:49.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3317000000000001e-05 2024-09-18T00:58:49.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:49.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521422+0000", 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480999999999996e-05 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521496+0000", 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4909999999999999e-05 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.442 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:49.444 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:49.445 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:49.441+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:49.545 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:49.537+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:49.611 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:49.611 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:49.611 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:49.611 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:49.625 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:49.669 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:49.665+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:49.742 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:49.742 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:49.742 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:49.743 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:49.743 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:49.747 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:49.741+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:49.756 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:49.813 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:58:49.858 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:49.853+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:49.922 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:49.923 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:49.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:49.923 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:49.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.923 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:49.923 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:49.923 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.510673081, 2024-09-18T00:58:49.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T00:58:49.923 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.923 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.923 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T00:58:49.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:49.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:26797 3.0 3:03a9420d:::rbd_data.117c4c7db1a0.0000000000000071:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.732246281999998, 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073789632999999993, 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.929 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 26797 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187640+0000", 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187653+0000", 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2357e-05 2024-09-18T00:58:49.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187684+0000", 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1582999999999999e-05 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187699+0000", 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5003e-05 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187806+0000", 2024-09-18T00:58:49.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106776 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.188026+0000", 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021975799999999999 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7", 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.201663+0000", 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013637768 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:49.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261226+0000", 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059562916 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261261+0000", 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4360000000000003e-05 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261354+0000", 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3041999999999995e-05 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261427+0000", 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3553000000000005e-05 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64346 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330994696000001, 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048905196999999997, 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.935 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64346 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588890+0000", 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3e-07 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588893+0000", 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.106e-06 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588904+0000", 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1037e-05 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588911+0000", 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2729999999999997e-06 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637426+0000", 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048514483999999997 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637537+0000", 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110901 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637795+0000", 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025776599999999998 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64347 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330962823, 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049469502999999998, 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64347 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.940 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588922+0000", 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4300000000000003e-07 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588925+0000", 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588934+0000", 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0329999999999997e-06 2024-09-18T00:58:49.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588946+0000", 2024-09-18T00:58:49.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1547e-05 2024-09-18T00:58:49.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637872+0000", 2024-09-18T00:58:49.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048926202000000002 2024-09-18T00:58:49.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638047+0000", 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174566 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638391+0000", 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034401499999999998 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.944 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64348 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330921953000001, 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050239487999999999, 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64348 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588963+0000", 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.133e-06 2024-09-18T00:58:49.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588969+0000", 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4229999999999996e-06 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588996+0000", 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7396e-05 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589163+0000", 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016677900000000001 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638498+0000", 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049334743 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638715+0000", 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000217414 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639202+0000", 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048660000000000001 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64349 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330693048000001, 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050607656000000001, 2024-09-18T00:58:49.949 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64349 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589192+0000", 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.536e-06 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589201+0000", 2024-09-18T00:58:49.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-06 2024-09-18T00:58:49.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589222+0000", 2024-09-18T00:58:49.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-18T00:58:49.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589241+0000", 2024-09-18T00:58:49.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8960000000000001e-05 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639285+0000", 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050043892999999999 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639457+0000", 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017286299999999999 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639799+0000", 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034117400000000002 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64350 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330622703, 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051028502000000003, 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.954 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64350 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589262+0000", 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1570000000000001e-06 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589280+0000", 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7943000000000001e-05 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589299+0000", 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8940000000000002e-05 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589316+0000", 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6654e-05 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639872+0000", 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050556151000000001 2024-09-18T00:58:49.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639988+0000", 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115423 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640290+0000", 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030223400000000002 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64351 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330546963, 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051389975999999997, 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64351 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589338+0000", 2024-09-18T00:58:49.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589345+0000", 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589363+0000", 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8346e-05 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589381+0000", 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7357e-05 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640373+0000", 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050992347 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640504+0000", 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000131011 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640727+0000", 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022310799999999999 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64352 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330483434000001, 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051932649999999997, 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.963 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64352 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589402+0000", 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.237e-06 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589408+0000", 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2770000000000002e-06 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589426+0000", 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8006e-05 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589441+0000", 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4934e-05 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640803+0000", 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051361619999999997 2024-09-18T00:58:49.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640986+0000", 2024-09-18T00:58:49.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018303199999999999 2024-09-18T00:58:49.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641333+0000", 2024-09-18T00:58:49.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034754399999999998 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64353 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330425040000002, 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052565148999999999, 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.968 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64353 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:49.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589460+0000", 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4300000000000001e-06 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589466+0000", 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.446e-06 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589485+0000", 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8726999999999999e-05 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589499+0000", 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.402e-05 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641427+0000", 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051927945000000003 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641601+0000", 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174859 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642024+0000", 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000422722 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.972 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64354 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330350340999999, 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053055607999999997, 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64354 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589535+0000", 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3430000000000001e-06 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589542+0000", 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589561+0000", 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9207000000000001e-05 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589577+0000", 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5933000000000001e-05 2024-09-18T00:58:49.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642111+0000", 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052533799999999999 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642271+0000", 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015998499999999999 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642589+0000", 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031818999999999999 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64355 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330286305000001, 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053570327000000001, 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.977 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64355 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:49.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589599+0000", 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589605+0000", 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8830000000000003e-06 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589622+0000", 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.732e-05 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589636+0000", 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3703000000000001e-05 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642672+0000", 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053036204000000003 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642828+0000", 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155451 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643168+0000", 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034051900000000001 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.981 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64356 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330228792, 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054203299000000003, 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64356 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589656+0000", 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-18T00:58:49.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589662+0000", 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9800000000000003e-06 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589681+0000", 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8300000000000001e-05 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589695+0000", 2024-09-18T00:58:49.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3997e-05 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643249+0000", 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053554101999999999 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643423+0000", 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174062 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643859+0000", 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043562499999999998 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64357 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330170842000001, 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054239849, 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.986 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64357 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589714+0000", 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.266e-06 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589720+0000", 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2340000000000002e-06 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589738+0000", 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7932999999999999e-05 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589752+0000", 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4867e-05 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643934+0000", 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054181721000000002 2024-09-18T00:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643953+0000", 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8828e-05 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64358 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330112963000001, 2024-09-18T00:58:49.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054261125, 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64358 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589772+0000", 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1170000000000001e-06 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589778+0000", 2024-09-18T00:58:49.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8429999999999998e-06 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589799+0000", 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0727e-05 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589813+0000", 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4403000000000001e-05 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644021+0000", 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054207784000000002 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644032+0000", 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1251e-05 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64359 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.994 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.330050997000001, 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286586999999997, 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64359 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589834+0000", 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589841+0000", 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.956e-06 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589864+0000", 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3124000000000001e-05 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589879+0000", 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.486e-05 2024-09-18T00:58:49.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:49.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644099+0000", 2024-09-18T00:58:49.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054219429 2024-09-18T00:58:49.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:49.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644120+0000", 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1004000000000001e-05 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64360 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.329986803000001, 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054290699999999997, 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:49.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64360 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:50.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589898+0000", 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3600000000000002e-07 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589903+0000", 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4669999999999998e-06 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589913+0000", 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0463e-05 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589921+0000", 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7959999999999996e-06 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644173+0000", 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054251903999999997 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644188+0000", 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5133999999999999e-05 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64361 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.329953324000002, 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054320101000000003, 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.003 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64361 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5000000000000002e-07 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589935+0000", 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2799999999999999e-06 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589944+0000", 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589954+0000", 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0036e-05 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644240+0000", 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286273000000003 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644251+0000", 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0392e-05 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64362 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.329920177999998, 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054340196, 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.007 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64362 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589978+0000", 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3570000000000001e-05 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589991+0000", 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3196999999999999e-05 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590000+0000", 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2800000000000003e-06 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644294+0000", 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054294873 2024-09-18T00:58:50.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644304+0000", 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6560000000000004e-06 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64363 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.329874147999998, 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054356660000000001, 2024-09-18T00:58:50.011 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64363 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590011+0000", 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7400000000000003e-07 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590014+0000", 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:58:50.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590024+0000", 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5670000000000001e-06 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590031+0000", 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3830000000000003e-06 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644354+0000", 2024-09-18T00:58:50.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054323429999999999 2024-09-18T00:58:50.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644367+0000", 2024-09-18T00:58:50.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.217e-05 2024-09-18T00:58:50.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.015 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.015 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.015 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:50.017 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:50.017 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:50.009+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:50.118 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:50.113+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:50.146 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:50.146 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:50.146 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:50.146 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:50.160 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:50.219 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:50.213+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:50.287 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:50.287 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:50.287 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:50.287 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:50.287 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:50.302 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:50.345 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:50.341+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:50.420 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:50.413+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:50.468 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:50.468 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:50.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:50.468 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:50.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.468 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.376848072999998, 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T00:58:50.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T00:58:50.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.985953121999998, 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.473 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T00:58:50.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78503 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6372290009999997, 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053463986999999998, 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78503 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.478 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:50.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6899999999999997e-07 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828236+0000", 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.853e-06 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828243+0000", 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4359999999999996e-06 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828257+0000", 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3862999999999999e-05 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881475+0000", 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053217299000000003 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881569+0000", 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4744e-05 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881697+0000", 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012722300000000001 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78504 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 897024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6371950970000002, 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053679381999999998, 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.482 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78504 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828572+0000", 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030532900000000002 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828577+0000", 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8579999999999999e-06 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828596+0000", 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9414000000000001e-05 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828604+0000", 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0190000000000006e-06 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881742+0000", 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053137864 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881822+0000", 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0346000000000006e-05 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881946+0000", 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123552 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.486 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78505 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6368414089999996, 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053688825000000003, 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78505 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828656+0000", 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6273999999999999e-05 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828660+0000", 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.76e-06 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828670+0000", 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5279999999999992e-06 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828678+0000", 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8560000000000007e-06 2024-09-18T00:58:50.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881994+0000", 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053316364999999998 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882203+0000", 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020851600000000001 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882309+0000", 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106526 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78506 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6367725990000004, 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032405999999998, 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.491 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78506 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:50.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828976+0000", 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028726700000000001 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828983+0000", 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.246e-06 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828996+0000", 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3125999999999999e-05 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829003+0000", 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6899999999999992e-06 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882345+0000", 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053341462999999999 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882423+0000", 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8338999999999997e-05 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882721+0000", 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029827500000000003 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.495 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78507 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6364453379999997, 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053936698999999998, 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78507 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829065+0000", 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8863999999999998e-05 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829069+0000", 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9160000000000003e-06 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829080+0000", 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1087000000000001e-05 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829088+0000", 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2609999999999993e-06 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882766+0000", 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053678037999999997 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882847+0000", 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0131999999999998e-05 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882953+0000", 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106401 2024-09-18T00:58:50.499 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.500 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.500 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.500 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78508 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 978944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.500 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:50.500 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6363619199999997, 2024-09-18T00:58:50.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053997636000000002, 2024-09-18T00:58:50.500 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78508 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830739+0000", 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001639738 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830745+0000", 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8869999999999997e-06 2024-09-18T00:58:50.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830755+0000", 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8830000000000001e-06 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830765+0000", 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7589999999999992e-06 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882990+0000", 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052225252 2024-09-18T00:58:50.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.883040+0000", 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9848000000000001e-05 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.883097+0000", 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7268999999999997e-05 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78514 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1101824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6345542909999997, 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053776592999999998, 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78514 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.505 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830908+0000", 2024-09-18T00:58:50.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830928+0000", 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9898000000000002e-05 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830940+0000", 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2483999999999999e-05 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830955+0000", 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5143999999999999e-05 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884176+0000", 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053221151000000001 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884581+0000", 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040494199999999999 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884684+0000", 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102476 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78515 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1122304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6344965760000001, 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053953972000000003, 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.509 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78515 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:50.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830966+0000", 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4799999999999998e-07 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830968+0000", 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6390000000000001e-06 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830975+0000", 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4790000000000001e-06 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830980+0000", 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4269999999999999e-06 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884731+0000", 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053750345999999997 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884785+0000", 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.49e-05 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884919+0000", 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133533 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.513 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78516 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6344730079999996, 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054090795999999997, 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78516 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830991+0000", 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.221e-06 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830998+0000", 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5409999999999997e-06 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831003+0000", 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2390000000000002e-06 2024-09-18T00:58:50.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884953+0000", 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053950117999999998 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885000+0000", 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6555999999999999e-05 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885079+0000", 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9702000000000004e-05 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78517 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1163264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6344520720000002, 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298990999999998, 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.518 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78517 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4400000000000001e-07 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831018+0000", 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2490000000000001e-06 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831025+0000", 2024-09-18T00:58:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831031+0000", 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.463e-06 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885130+0000", 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054099371 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885207+0000", 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7292000000000005e-05 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885309+0000", 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010129899999999999 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78518 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1183744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:50.522 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6344222930000001, 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054532499999999998, 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78518 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:50.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831040+0000", 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3099999999999997e-07 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831042+0000", 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2879999999999999e-06 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831050+0000", 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.464e-06 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831055+0000", 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0819999999999998e-06 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885354+0000", 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298739999999998 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885428+0000", 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4567999999999995e-05 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885572+0000", 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014372699999999999 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.526 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78519 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1204224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6344004439999997, 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054707261, 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.527 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78519 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831062+0000", 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831064+0000", 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0360000000000001e-06 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831069+0000", 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4149999999999998e-06 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831075+0000", 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2249999999999997e-06 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885602+0000", 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054526893 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885673+0000", 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0865000000000007e-05 2024-09-18T00:58:50.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885768+0000", 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5422999999999998e-05 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78520 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1224704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6343799880000001, 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054906695999999998, 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.531 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78520 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831084+0000", 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2960000000000001e-06 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831091+0000", 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831095+0000", 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.792e-06 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885802+0000", 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054706795000000003 2024-09-18T00:58:50.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885884+0000", 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2297000000000002e-05 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885988+0000", 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010393199999999999 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.535 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78521 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6343351149999998, 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055066616999999998, 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78521 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831127+0000", 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.99e-07 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831130+0000", 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6819999999999999e-06 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831137+0000", 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3849999999999996e-06 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831145+0000", 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7360000000000001e-06 2024-09-18T00:58:50.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886020+0000", 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054874700999999998 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886096+0000", 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6234000000000002e-05 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886193+0000", 2024-09-18T00:58:50.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7280000000000004e-05 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78522 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.634309354, 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055137938999999997, 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.540 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78522 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831153+0000", 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.37e-07 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831155+0000", 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4420000000000001e-06 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831161+0000", 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:58:50.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831166+0000", 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8749999999999999e-06 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886279+0000", 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055112337999999997 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886290+0000", 2024-09-18T00:58:50.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1527e-05 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78523 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6342886740000004, 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055171425000000003, 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:50.544 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78523 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0900000000000002e-07 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831176+0000", 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4830000000000002e-06 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831183+0000", 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0679999999999999e-06 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831189+0000", 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886331+0000", 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055142669999999998 2024-09-18T00:58:50.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886344+0000", 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2918999999999999e-05 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78524 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6342665580000002, 2024-09-18T00:58:50.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055194430000000003, 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78524 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831196+0000", 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1200000000000003e-07 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831198+0000", 2024-09-18T00:58:50.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831204+0000", 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2740000000000004e-06 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831209+0000", 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0900000000000004e-06 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886381+0000", 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055172142 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886389+0000", 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2150000000000001e-06 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78525 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.6342460450000003, 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055206535000000001, 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78525 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:50.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832358+0000", 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001142531 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:50.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832364+0000", 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832375+0000", 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1381e-05 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832382+0000", 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9800000000000001e-06 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886415+0000", 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032678000000001 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886422+0000", 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4449999999999999e-06 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:50.556 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:50.557 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:50.558 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:50.559 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:50.553+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:50.660 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:50.653+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:50.706 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:50.706 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:50.723 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:50.723 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:50.728 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:50.728 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:50.739 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:50.739 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:50.742 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:50.756 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:50.756 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:50.774 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:50.775 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:50.793 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:50.789+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:50.794 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:50.794 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:50.807 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:50.823 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:50.853 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:50.861 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:50.857+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:50.875 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:50.875 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:50.890 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:50.962 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:50.957+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:51.058 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:51.058 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:51.059 INFO:teuthology.orchestra.run.smithi042.stdout:3612 read 0x147ebee thru 0x14846da (0x5aed bytes) 2024-09-18T00:58:51.063 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:51.057+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:51.075 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:51.075 INFO:teuthology.orchestra.run.smithi042.stdout:3613 writesame 0xc0b42f thru 0xc13e10 data_size 0x6f(0x89e2 bytes) 2024-09-18T00:58:51.165 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:51.157+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:51.238 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:51.238 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:51.256 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:51.298 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:51.293+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:51.368 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:51.361+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:51.402 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:51.402 INFO:teuthology.orchestra.run.smithi042.stdout:3616 write 0x78be37 thru 0x78d6ca (0x1894 bytes) 2024-09-18T00:58:51.443 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:51.443 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:51.460 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:51.492 INFO:teuthology.orchestra.run.smithi042.stdout:3617 read 0x7c8bd04 thru 0x7c8d6d4 (0x19d1 bytes) 2024-09-18T00:58:51.492 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:51.485+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:51.492 INFO:teuthology.orchestra.run.smithi042.stdout:3618 write 0x6fa7d94 thru 0x6fae129 (0x6396 bytes) 2024-09-18T00:58:51.510 INFO:teuthology.orchestra.run.smithi042.stdout:3619 writesame 0x683e315 thru 0x6846457 data_size 0x1(0x8143 bytes) 2024-09-18T00:58:51.535 INFO:teuthology.orchestra.run.smithi042.stdout:3620 punch from 0x41cf3ea to 0x41d3de9, (0x49ff bytes) 2024-09-18T00:58:51.536 INFO:teuthology.orchestra.run.smithi042.stdout:3622 punch from 0x4ba873 to 0x4c4668, (0x9df5 bytes) 2024-09-18T00:58:51.536 INFO:teuthology.orchestra.run.smithi042.stdout:3623 write 0x55ea4bc thru 0x55edb8c (0x36d1 bytes) 2024-09-18T00:58:51.544 INFO:teuthology.orchestra.run.smithi042.stdout:3624 read 0x83abd0 thru 0x842cab (0x80dc bytes) 2024-09-18T00:58:51.546 INFO:teuthology.orchestra.run.smithi042.stdout:3626 write 0x4bd4a66 thru 0x4bd5066 (0x601 bytes) 2024-09-18T00:58:51.556 INFO:teuthology.orchestra.run.smithi042.stdout:3627 writesame 0x9b6d90d thru 0x9b78871 data_size 0x9(0xaf65 bytes) 2024-09-18T00:58:51.567 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:51.561+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:51.577 INFO:teuthology.orchestra.run.smithi042.stdout:3628 punch from 0x963229d to 0x96407e9, (0xe54c bytes) 2024-09-18T00:58:51.577 INFO:teuthology.orchestra.run.smithi042.stdout:3629 read 0x1c9dcba thru 0x1ca7809 (0x9b50 bytes) 2024-09-18T00:58:51.601 INFO:teuthology.orchestra.run.smithi042.stdout:3630 read 0x71e6859 thru 0x71e8b92 (0x233a bytes) 2024-09-18T00:58:51.617 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:51.617 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:51.632 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:51.637 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:51.637 INFO:teuthology.orchestra.run.smithi042.stdout:3632 punch from 0x5d3a605 to 0x5d3f6e4, (0x50df bytes) 2024-09-18T00:58:51.637 INFO:teuthology.orchestra.run.smithi042.stdout:3633 punch from 0x1c73155 to 0x1c7c637, (0x94e2 bytes) 2024-09-18T00:58:51.637 INFO:teuthology.orchestra.run.smithi042.stdout:3634 read 0x3b3e722 thru 0x3b483e3 (0x9cc2 bytes) 2024-09-18T00:58:51.639 INFO:teuthology.orchestra.run.smithi042.stdout:3635 write 0x7748dea thru 0x774df82 (0x5199 bytes) 2024-09-18T00:58:51.668 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:51.661+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:51.768 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:51.761+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:51.769 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:51.769 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:51.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:51.796 INFO:teuthology.orchestra.run.smithi042.stdout:3637 write 0x70b314d thru 0x70b6d26 (0x3bda bytes) 2024-09-18T00:58:51.801 INFO:teuthology.orchestra.run.smithi042.stdout:3638 write 0x3ebf3a2 thru 0x3ec245c (0x30bb bytes) 2024-09-18T00:58:51.803 INFO:teuthology.orchestra.run.smithi042.stdout:3639 writesame 0x1075dd9 thru 0x107706e data_size 0xd(0x1296 bytes) 2024-09-18T00:58:51.806 INFO:teuthology.orchestra.run.smithi042.stdout:3641 writesame 0x51650dc thru 0x5173752 data_size 0x1(0xe677 bytes) 2024-09-18T00:58:51.831 INFO:teuthology.orchestra.run.smithi042.stdout:3642 read 0x75ac126 thru 0x75ba1bb (0xe096 bytes) 2024-09-18T00:58:51.836 INFO:teuthology.orchestra.run.smithi042.stdout:3643 punch from 0x50d1b33 to 0x50d43a2, (0x286f bytes) 2024-09-18T00:58:51.836 INFO:teuthology.orchestra.run.smithi042.stdout:3644 punch from 0x701a0f0 to 0x7024e74, (0xad84 bytes) 2024-09-18T00:58:51.836 INFO:teuthology.orchestra.run.smithi042.stdout:3645 writesame 0x3ef8d41 thru 0x3efc5de data_size 0x1(0x389e bytes) 2024-09-18T00:58:51.869 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:51.865+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:51.906 INFO:teuthology.orchestra.run.smithi042.stdout:3646 punch from 0x32ee6a0 to 0x32f7829, (0x9189 bytes) 2024-09-18T00:58:51.906 INFO:teuthology.orchestra.run.smithi042.stdout:3647 read 0x9982d00 thru 0x998d537 (0xa838 bytes) 2024-09-18T00:58:51.908 INFO:teuthology.orchestra.run.smithi042.stdout:3649 write 0x5250b76 thru 0x52601ec (0xf677 bytes) 2024-09-18T00:58:51.921 INFO:teuthology.orchestra.run.smithi042.stdout:3651 read 0x8b26352 thru 0x8b34fbb (0xec6a bytes) 2024-09-18T00:58:51.934 INFO:teuthology.orchestra.run.smithi042.stdout:3652 trunc from 0xaa257ed to 0x213ffc9 2024-09-18T00:58:51.945 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:51.946 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:51.962 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:52.010 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:52.005+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:52.070 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:52.065+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:52.121 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:52.121 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:52.134 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:52.178 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:52.173+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:52.273 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:52.265+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:52.294 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:52.294 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:52.307 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:52.394 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:52.389+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:52.469 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:52.469 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:52.474 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:52.469+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:52.482 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:52.575 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:52.569+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:52.668 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:52.668 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:52.677 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:52.673+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:52.684 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:52.778 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:52.773+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:52.851 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:58:52.852 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:52.853 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:52.867 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:52.875 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:52.878 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:52.873+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:52.979 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:52.973+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:53.032 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:53.032 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:53.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:53.059 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:53.060 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:53.076 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:53.076 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:53.081 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:53.077+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:53.093 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:53.094 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:53.112 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:53.113 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:53.134 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:53.134 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:53.156 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:53.156 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:53.177 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:53.177 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:53.178 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:58:53.178 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:58:53.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:58:53.178 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:58:53.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.178 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:58:53.178 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:53.178 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 85.862358783000005, 2024-09-18T00:58:53.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.101264911, 2024-09-18T00:58:53.179 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:53.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4226", 2024-09-18T00:58:53.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2155402750", 2024-09-18T00:58:53.179 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:58:53.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.179 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313758+0000", 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313760+0000", 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.612e-06 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313768+0000", 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9249999999999995e-06 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313786+0000", 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8253999999999998e-05 2024-09-18T00:58:53.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313803+0000", 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7388999999999999e-05 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313935+0000", 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013180399999999999 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.313954+0000", 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8768999999999999e-05 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.411874+0000", 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.097920646 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412292+0000", 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041777699999999998 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.412675+0000", 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038262300000000001 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414019+0000", 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013436699999999999 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414872+0000", 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00085320999999999997 2024-09-18T00:58:53.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:53.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.414955+0000", 2024-09-18T00:58:53.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3405999999999994e-05 2024-09-18T00:58:53.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:27.415023+0000", 2024-09-18T00:58:53.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7825999999999997e-05 2024-09-18T00:58:53.185 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.185 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:15500 3.11 3:8ed6a97b:::rbd_data.117c53e529c1.000000000000001a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.361598763000003, 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046818956000000002, 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 15500 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814518+0000", 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814521+0000", 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.977e-06 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814536+0000", 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.487e-05 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814572+0000", 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5567e-05 2024-09-18T00:58:53.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814588+0000", 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6546e-05 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814675+0000", 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6675999999999995e-05 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.814839+0000", 2024-09-18T00:58:53.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016414199999999999 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.819908+0000", 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0050695280000000002 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861139+0000", 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.041230771999999999 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861208+0000", 2024-09-18T00:58:53.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8722999999999999e-05 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861277+0000", 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9246999999999996e-05 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:52.861337+0000", 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9908000000000003e-05 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87779 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1404928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.534897+0000", 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.641220176, 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045551541000000001, 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87779 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.192 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534897+0000", 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534897+0000", 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534897+0000", 2024-09-18T00:58:53.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534898+0000", 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0759999999999999e-06 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534901+0000", 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1810000000000001e-06 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534904+0000", 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6620000000000001e-06 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580264+0000", 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045360294000000002 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580350+0000", 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5556000000000006e-05 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580448+0000", 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8548999999999994e-05 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87780 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.534907+0000", 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6412098320000004, 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045657725000000003, 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.196 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87780 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534907+0000", 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534907+0000", 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534907+0000", 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.41e-07 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534908+0000", 2024-09-18T00:58:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534912+0000", 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.681e-06 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534916+0000", 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.442e-06 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580463+0000", 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045546924000000003 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580510+0000", 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6390999999999997e-05 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580565+0000", 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4968e-05 2024-09-18T00:58:53.200 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.201 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.201 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87781 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1445888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.201 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.534943+0000", 2024-09-18T00:58:53.201 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6411740610000001, 2024-09-18T00:58:53.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045839058000000002, 2024-09-18T00:58:53.201 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.201 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.201 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.204 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87781 2024-09-18T00:58:53.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.204 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534943+0000", 2024-09-18T00:58:53.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534943+0000", 2024-09-18T00:58:53.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534943+0000", 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2399999999999999e-07 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534946+0000", 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4980000000000001e-06 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534953+0000", 2024-09-18T00:58:53.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.985e-06 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534958+0000", 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5860000000000004e-06 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580602+0000", 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045643209999999997 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580684+0000", 2024-09-18T00:58:53.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2539000000000004e-05 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580782+0000", 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7816e-05 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87782 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1466368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.534966+0000", 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6411506879999997, 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045923248, 2024-09-18T00:58:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87782 2024-09-18T00:58:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534966+0000", 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534966+0000", 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534967+0000", 2024-09-18T00:58:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.58e-07 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534969+0000", 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1940000000000001e-06 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534977+0000", 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7679999999999997e-06 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534983+0000", 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7180000000000001e-06 2024-09-18T00:58:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580798+0000", 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045814663999999998 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580837+0000", 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9316000000000001e-05 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580889+0000", 2024-09-18T00:58:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.223e-05 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87783 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1486848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.534990+0000", 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6411263839999997, 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046128573999999999, 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87783 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534990+0000", 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534990+0000", 2024-09-18T00:58:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534991+0000", 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1399999999999997e-07 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.534993+0000", 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535004+0000", 2024-09-18T00:58:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0983e-05 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535012+0000", 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0199999999999994e-06 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.580928+0000", 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045915456 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581007+0000", 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9556999999999997e-05 2024-09-18T00:58:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581119+0000", 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111864 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87784 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1507328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535020+0000", 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6410964379999999, 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046227183999999998, 2024-09-18T00:58:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87784 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535020+0000", 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535020+0000", 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535021+0000", 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1600000000000002e-07 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535023+0000", 2024-09-18T00:58:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2929999999999999e-06 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535030+0000", 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9859999999999997e-06 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535035+0000", 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1499999999999998e-06 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581153+0000", 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046117461999999998 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581195+0000", 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2076999999999997e-05 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581248+0000", 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-05 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87785 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1527808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535042+0000", 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6410748980000003, 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046410976, 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87785 2024-09-18T00:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535042+0000", 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535042+0000", 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535042+0000", 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9499999999999998e-07 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535044+0000", 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0940000000000002e-06 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535052+0000", 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0229999999999992e-06 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535061+0000", 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9509999999999995e-06 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581282+0000", 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046221048000000001 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581357+0000", 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4912000000000004e-05 2024-09-18T00:58:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581453+0000", 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5552999999999998e-05 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87786 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1548288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535069+0000", 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6410474759999998, 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046523549999999997, 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87786 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535069+0000", 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535069+0000", 2024-09-18T00:58:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535070+0000", 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.96e-07 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535097+0000", 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7526999999999999e-05 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535106+0000", 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2339999999999994e-06 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535112+0000", 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581490+0000", 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.04637819 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581534+0000", 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4283000000000003e-05 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581593+0000", 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8888999999999998e-05 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87787 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535124+0000", 2024-09-18T00:58:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6409933079999997, 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046685899000000003, 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87787 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535124+0000", 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535124+0000", 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535124+0000", 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8100000000000003e-07 2024-09-18T00:58:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535127+0000", 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5459999999999998e-06 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535135+0000", 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4020000000000002e-06 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535141+0000", 2024-09-18T00:58:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6559999999999997e-06 2024-09-18T00:58:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581633+0000", 2024-09-18T00:58:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046492646999999998 2024-09-18T00:58:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581717+0000", 2024-09-18T00:58:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3815999999999998e-05 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581809+0000", 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2250999999999995e-05 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87788 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1589248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535148+0000", 2024-09-18T00:58:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6409684440000003, 2024-09-18T00:58:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046785392000000002, 2024-09-18T00:58:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87788 2024-09-18T00:58:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535148+0000", 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535148+0000", 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535277+0000", 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000129071 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535280+0000", 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9000000000000002e-06 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535287+0000", 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.618e-06 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535291+0000", 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2250000000000002e-06 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581841+0000", 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046549556999999998 2024-09-18T00:58:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581882+0000", 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1273000000000003e-05 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581934+0000", 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1748e-05 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87789 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1609728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535308+0000", 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6408093450000001, 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046832960999999999, 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87789 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535308+0000", 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535308+0000", 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535418+0000", 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000110726 2024-09-18T00:58:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535421+0000", 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4609999999999998e-06 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535428+0000", 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535435+0000", 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5089999999999999e-06 2024-09-18T00:58:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.581970+0000", 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046535853000000002 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582047+0000", 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6457000000000003e-05 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582140+0000", 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3597999999999994e-05 2024-09-18T00:58:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87790 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 1630208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535445+0000", 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6406717349999997, 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046815748999999997, 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87790 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535445+0000", 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535445+0000", 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535640+0000", 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019530599999999999 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535644+0000", 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6320000000000001e-06 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535657+0000", 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2957e-05 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535662+0000", 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4970000000000003e-06 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582171+0000", 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046509485000000003 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582212+0000", 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0844999999999999e-05 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582261+0000", 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9026999999999997e-05 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87791 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.535673+0000", 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6404440219999996, 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046637830999999998, 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87791 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535673+0000", 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.535673+0000", 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536311+0000", 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063850300000000001 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536316+0000", 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.408e-06 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536325+0000", 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7380000000000006e-06 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536338+0000", 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2099000000000001e-05 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582299+0000", 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045961850999999998 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582311+0000", 2024-09-18T00:58:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1232e-05 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87792 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.536351+0000", 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6397660329999999, 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045989887, 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87792 2024-09-18T00:58:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536351+0000", 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536351+0000", 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536351+0000", 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536354+0000", 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6410000000000002e-06 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536361+0000", 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4229999999999999e-06 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536368+0000", 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.72e-06 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582334+0000", 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045966295999999997 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582341+0000", 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3029999999999996e-06 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87793 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.258 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.536393+0000", 2024-09-18T00:58:53.259 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6397239380000004, 2024-09-18T00:58:53.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045987987000000001, 2024-09-18T00:58:53.259 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:53.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87793 2024-09-18T00:58:53.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536393+0000", 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536393+0000", 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536393+0000", 2024-09-18T00:58:53.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9399999999999995e-07 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536396+0000", 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536405+0000", 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8820000000000007e-06 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536411+0000", 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1619999999999996e-06 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582374+0000", 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045962692999999999 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582381+0000", 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8399999999999997e-06 2024-09-18T00:58:53.262 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87794 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.536421+0000", 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6396960729999996, 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045991835000000002, 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87794 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536421+0000", 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536421+0000", 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536421+0000", 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2100000000000002e-07 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536424+0000", 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7659999999999999e-06 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536429+0000", 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7369999999999997e-06 2024-09-18T00:58:53.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536432+0000", 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1520000000000001e-06 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582406+0000", 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045974358 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582413+0000", 2024-09-18T00:58:53.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000002e-06 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87795 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.536446+0000", 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.639670755, 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045992551999999999, 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87795 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536446+0000", 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536446+0000", 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536664+0000", 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000217636 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536668+0000", 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9269999999999998e-06 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536676+0000", 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0960000000000001e-06 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536683+0000", 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7569999999999998e-06 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582433+0000", 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045750199999999998 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582439+0000", 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9359999999999996e-06 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:87796 3.1d 3.19daefdd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:48.536693+0000", 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.639424236, 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045774065000000003, 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:58:53.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 87796 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536693+0000", 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.536693+0000", 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:58:53.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.537106+0000", 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000413075 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.537109+0000", 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:58:53.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.537117+0000", 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8909999999999995e-06 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.537121+0000", 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5249999999999999e-06 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582460+0000", 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045338090999999997 2024-09-18T00:58:53.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:58:53.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:58:53.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:58:53.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:48.582467+0000", 2024-09-18T00:58:53.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1080000000000004e-06 2024-09-18T00:58:53.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:58:53.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:58:53.275 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:58:53.277 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:53.277 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:53.278 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:53.278 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:53.273+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:53.378 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:53.373+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:53.410 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:53.411 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:53.411 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:53.411 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:53.424 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:53.479 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:53.477+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:53.544 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:53.545 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:53.558 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:53.580 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:53.573+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1672 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.508639549999998, 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083853480999999994, 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.677 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1672 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166697+0000", 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0920000000000001e-06 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166702+0000", 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0799999999999996e-06 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166718+0000", 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6167999999999998e-05 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166745+0000", 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7177000000000001e-05 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250382+0000", 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083636792000000001 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250476+0000", 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250549+0000", 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3282999999999996e-05 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1673 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.681 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.508572442000002, 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084151880999999998, 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1673 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166764+0000", 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166770+0000", 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5969999999999999e-06 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166786+0000", 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6439999999999998e-05 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166799+0000", 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2934e-05 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250605+0000", 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083806357999999997 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250724+0000", 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011821500000000001 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250915+0000", 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000191251 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.685 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1674 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.508472857000001, 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084239101999999996, 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1674 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.686 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166864+0000", 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3629999999999999e-06 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166883+0000", 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9083e-05 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166902+0000", 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8711000000000001e-05 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166915+0000", 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3375e-05 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250962+0000", 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084047254000000002 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251029+0000", 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6315000000000005e-05 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251102+0000", 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3001000000000001e-05 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.689 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1675 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1540096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.508401006, 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084536891000000003, 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.690 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1675 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166936+0000", 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2720000000000001e-06 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166941+0000", 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7370000000000003e-06 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166958+0000", 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166972+0000", 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3569e-05 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251159+0000", 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084187467000000002 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251300+0000", 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014059200000000001 2024-09-18T00:58:53.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251471+0000", 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017168000000000001 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1676 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.508348126000001, 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084738637000000006, 2024-09-18T00:58:53.694 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1676 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166988+0000", 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166993+0000", 2024-09-18T00:58:53.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8339999999999998e-06 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167009+0000", 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6028e-05 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167022+0000", 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2675e-05 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251517+0000", 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084494975999999999 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251576+0000", 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8644999999999998e-05 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251726+0000", 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015039 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.698 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.699 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1677 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.699 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:53.699 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.508296454999993, 2024-09-18T00:58:53.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085055403000000002, 2024-09-18T00:58:53.699 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.699 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.699 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.699 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.699 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1677 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167040+0000", 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167046+0000", 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167062+0000", 2024-09-18T00:58:53.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6354e-05 2024-09-18T00:58:53.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167075+0000", 2024-09-18T00:58:53.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2965e-05 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251789+0000", 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084713940000000001 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251916+0000", 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000127176 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252094+0000", 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017833599999999999 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1678 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.508244692000005, 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085316746999999998, 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.704 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1678 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167092+0000", 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167097+0000", 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7029999999999997e-06 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167113+0000", 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167125+0000", 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2517000000000001e-05 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252157+0000", 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085031933000000004 2024-09-18T00:58:53.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252277+0000", 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119708 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252407+0000", 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013031800000000001 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1679 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.508193973000004, 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085557195000000003, 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1679 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167143+0000", 2024-09-18T00:58:53.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:58:53.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167170+0000", 2024-09-18T00:58:53.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7523000000000001e-05 2024-09-18T00:58:53.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167189+0000", 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8530999999999999e-05 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167215+0000", 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6676999999999999e-05 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252445+0000", 2024-09-18T00:58:53.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085229416000000002 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252529+0000", 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4740000000000005e-05 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252699+0000", 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016920800000000001 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.713 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.714 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1680 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.714 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:53.714 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.508100515999999, 2024-09-18T00:58:53.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085810110999999994, 2024-09-18T00:58:53.714 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.714 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.714 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.714 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1680 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:53.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167236+0000", 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.048e-06 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167242+0000", 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6459999999999998e-06 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167259+0000", 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.696e-05 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167271+0000", 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.292e-05 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252839+0000", 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085567292000000003 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252924+0000", 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5511000000000002e-05 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253045+0000", 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120734 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.718 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1681 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.507994420000003, 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085887343000000005, 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1681 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167342+0000", 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167348+0000", 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4519999999999996e-06 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167372+0000", 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4261999999999999e-05 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167400+0000", 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-18T00:58:53.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253091+0000", 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085690895000000003 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253154+0000", 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3262000000000005e-05 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253228+0000", 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4318000000000002e-05 2024-09-18T00:58:53.722 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1682 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.507916339999994, 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.086061112999999995, 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.723 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1682 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168114+0000", 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00069532600000000002 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168123+0000", 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8319999999999995e-06 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168147+0000", 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3536999999999998e-05 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168163+0000", 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6317e-05 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253274+0000", 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085111390999999995 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253362+0000", 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7384000000000002e-05 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253480+0000", 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011832600000000001 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.727 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1683 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1867776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.507139209000002, 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085456638000000001, 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1683 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169938+0000", 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017419079999999999 2024-09-18T00:58:53.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169951+0000", 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2534999999999999e-05 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169990+0000", 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9004999999999997e-05 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170011+0000", 2024-09-18T00:58:53.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.156e-05 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253534+0000", 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083522759000000002 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253585+0000", 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1316999999999997e-05 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253653+0000", 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7553999999999997e-05 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1685 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.505228775999996, 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083900785000000005, 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.732 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1685 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170108+0000", 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.229e-06 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170114+0000", 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0079999999999999e-06 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170132+0000", 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7734e-05 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170145+0000", 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2977e-05 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253866+0000", 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083721208000000005 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253934+0000", 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.834e-05 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254007+0000", 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3288999999999996e-05 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1686 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.736 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.505173565000007, 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084083526000000006, 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1686 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:53.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170163+0000", 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1909999999999999e-06 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170180+0000", 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7323e-05 2024-09-18T00:58:53.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170198+0000", 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7723e-05 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170212+0000", 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3645999999999999e-05 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254050+0000", 2024-09-18T00:58:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083838115000000005 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254136+0000", 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6106999999999994e-05 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254245+0000", 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010942099999999999 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1687 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.505105263000004, 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084186574, 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1687 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170647+0000", 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416649 2024-09-18T00:58:53.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170655+0000", 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6510000000000007e-06 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170683+0000", 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7093999999999998e-05 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170701+0000", 2024-09-18T00:58:53.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8383e-05 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254283+0000", 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083582091999999997 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254339+0000", 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6187999999999998e-05 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254417+0000", 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7516999999999997e-05 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1688 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.504609548999994, 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083931026000000006, 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.745 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1688 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173553+0000", 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002827333 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173566+0000", 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3276e-05 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173603+0000", 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6177000000000003e-05 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173628+0000", 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5343000000000001e-05 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254460+0000", 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.080831745999999996 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254547+0000", 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7045000000000004e-05 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254657+0000", 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011010600000000001 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.900472347000004, 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:58:53.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:53.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:53.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87406 3.19 3:9d734bda:::rbd_data.117c8934add3.000000000000007c:head [stat,delete] snapc 1f=[1f] RETRY=8 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:58:53.755 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.2525492250000001, 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097024104999999999, 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87406 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:58:53.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425532+0000", 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027458309999999998 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425537+0000", 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2839999999999997e-06 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425551+0000", 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3682999999999999e-05 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425559+0000", 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5329999999999997e-06 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515035+0000", 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.089476411000000006 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515168+0000", 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013309699999999999 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515475+0000", 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030662199999999999 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519553+0000", 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040781259999999996 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519665+0000", 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112669 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519748+0000", 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2270999999999999e-05 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519810+0000", 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2577999999999996e-05 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87408 3.19 3:9f063cdd:::rbd_data.117c8934add3.000000000000007d:head [stat,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425586+0000", 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.2497491219999999, 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.094380550999999993, 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.761 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87408 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425586+0000", 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425586+0000", 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425587+0000", 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8599999999999998e-07 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425590+0000", 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6790000000000001e-06 2024-09-18T00:58:53.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425596+0000", 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3690000000000003e-06 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425600+0000", 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5549999999999996e-06 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.516522+0000", 2024-09-18T00:58:53.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.090921960999999996 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.516767+0000", 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000244202 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.517317+0000", 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055067199999999999 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519642+0000", 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023243790000000001 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519827+0000", 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184816 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519909+0000", 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2799000000000005e-05 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519967+0000", 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7533e-05 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87414 3.19 3:9c145b25:::rbd_data.117c8934add3.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.2497256859999997, 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095886693999999995, 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:53.767 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87414 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425618+0000", 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9810000000000003e-06 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425625+0000", 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0870000000000001e-06 2024-09-18T00:58:53.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425630+0000", 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.262e-06 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518417+0000", 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092787816999999995 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518593+0000", 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017569100000000001 2024-09-18T00:58:53.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518939+0000", 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034572299999999998 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521320+0000", 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023809619999999999 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521343+0000", 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3317000000000001e-05 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521422+0000", 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480999999999996e-05 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521496+0000", 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4909999999999999e-05 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:53.772 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:53.774 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:53.775 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:53.769+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:53.858 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:58:53.858 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:58:53.858 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-18T00:58:53.859 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-18T00:58:53.875 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:53.869+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:53.901 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:53.901 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:53.901 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:53.901 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:53.909 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:58:53.917 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:54.005 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:54.001+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:54.021 INFO:teuthology.orchestra.run.smithi042.stdout:3654 read 0x1809e9e thru 0x1814d37 (0xae9a bytes) 2024-09-18T00:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout:3655 punch from 0x6ec0c3 to 0x6f0788, (0x46c5 bytes) 2024-09-18T00:58:54.076 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:54.069+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:54.117 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:54.118 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:54.135 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:54.176 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:54.169+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:54.179 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:54.179 INFO:teuthology.orchestra.run.smithi042.stdout:3659 read 0x1416990 thru 0x141b0eb (0x475c bytes) 2024-09-18T00:58:54.200 INFO:teuthology.orchestra.run.smithi042.stdout:3661 read 0xdc8f64 thru 0xdcb200 (0x229d bytes) 2024-09-18T00:58:54.244 INFO:teuthology.orchestra.run.smithi042.stdout:3663 trunc from 0x213ffc9 to 0xdef6944 2024-09-18T00:58:54.253 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-18T00:58:54.245+0000 7fdb35eeb640 -1 mon.a@0(leader).osd e85 definitely_dead 0 2024-09-18T00:58:54.278 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:54.273+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:54.327 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:54.327 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:54.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:54.327 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:54.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.327 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.916833135000005, 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T00:58:54.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T00:58:54.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:26797 3.0 3:03a9420d:::rbd_data.117c4c7db1a0.0000000000000071:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.138406336000003, 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073789632999999993, 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 26797 2024-09-18T00:58:54.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.335 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:54.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:54.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187640+0000", 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187653+0000", 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2357e-05 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187684+0000", 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1582999999999999e-05 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187699+0000", 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5003e-05 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187806+0000", 2024-09-18T00:58:54.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106776 2024-09-18T00:58:54.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.188026+0000", 2024-09-18T00:58:54.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021975799999999999 2024-09-18T00:58:54.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7", 2024-09-18T00:58:54.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.201663+0000", 2024-09-18T00:58:54.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013637768 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261226+0000", 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059562916 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261261+0000", 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4360000000000003e-05 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:54.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261354+0000", 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3041999999999995e-05 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261427+0000", 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3553000000000005e-05 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64346 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.737154749999998, 2024-09-18T00:58:54.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048905196999999997, 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64346 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588890+0000", 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3e-07 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588893+0000", 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.106e-06 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588904+0000", 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1037e-05 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588911+0000", 2024-09-18T00:58:54.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2729999999999997e-06 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637426+0000", 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048514483999999997 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637537+0000", 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110901 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637795+0000", 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025776599999999998 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64347 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.737122877000001, 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049469502999999998, 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64347 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588922+0000", 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4300000000000003e-07 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588925+0000", 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588934+0000", 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0329999999999997e-06 2024-09-18T00:58:54.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588946+0000", 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1547e-05 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637872+0000", 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048926202000000002 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638047+0000", 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174566 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638391+0000", 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034401499999999998 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64348 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:54.349 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.737082007000001, 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050239487999999999, 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64348 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588963+0000", 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.133e-06 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588969+0000", 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4229999999999996e-06 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588996+0000", 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7396e-05 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589163+0000", 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016677900000000001 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638498+0000", 2024-09-18T00:58:54.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049334743 2024-09-18T00:58:54.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638715+0000", 2024-09-18T00:58:54.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000217414 2024-09-18T00:58:54.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639202+0000", 2024-09-18T00:58:54.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048660000000000001 2024-09-18T00:58:54.354 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.354 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.354 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.355 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64349 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.355 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:54.355 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736853102000001, 2024-09-18T00:58:54.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050607656000000001, 2024-09-18T00:58:54.355 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.355 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.355 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.355 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.355 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.355 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.355 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64349 2024-09-18T00:58:54.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.356 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:54.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:54.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589192+0000", 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.536e-06 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589201+0000", 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-06 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589222+0000", 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589241+0000", 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8960000000000001e-05 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639285+0000", 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050043892999999999 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639457+0000", 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017286299999999999 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639799+0000", 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034117400000000002 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.359 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64350 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736782757, 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051028502000000003, 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64350 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589262+0000", 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1570000000000001e-06 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589280+0000", 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7943000000000001e-05 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589299+0000", 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8940000000000002e-05 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589316+0000", 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6654e-05 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639872+0000", 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050556151000000001 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639988+0000", 2024-09-18T00:58:54.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115423 2024-09-18T00:58:54.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640290+0000", 2024-09-18T00:58:54.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030223400000000002 2024-09-18T00:58:54.364 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.364 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.364 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.365 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64351 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.365 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:54.365 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736707017000001, 2024-09-18T00:58:54.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051389975999999997, 2024-09-18T00:58:54.365 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.365 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.365 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.365 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.365 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.365 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64351 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589338+0000", 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589345+0000", 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589363+0000", 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8346e-05 2024-09-18T00:58:54.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589381+0000", 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7357e-05 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640373+0000", 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050992347 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640504+0000", 2024-09-18T00:58:54.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000131011 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640727+0000", 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022310799999999999 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64352 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736643487999999, 2024-09-18T00:58:54.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051932649999999997, 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64352 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589402+0000", 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.237e-06 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589408+0000", 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2770000000000002e-06 2024-09-18T00:58:54.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589426+0000", 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8006e-05 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589441+0000", 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4934e-05 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640803+0000", 2024-09-18T00:58:54.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051361619999999997 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640986+0000", 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018303199999999999 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641333+0000", 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034754399999999998 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64353 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736585093999999, 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052565148999999999, 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64353 2024-09-18T00:58:54.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589460+0000", 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4300000000000001e-06 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589466+0000", 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.446e-06 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589485+0000", 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8726999999999999e-05 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589499+0000", 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.402e-05 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641427+0000", 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051927945000000003 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641601+0000", 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174859 2024-09-18T00:58:54.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642024+0000", 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000422722 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64354 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736510395, 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053055607999999997, 2024-09-18T00:58:54.378 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64354 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589535+0000", 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3430000000000001e-06 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589542+0000", 2024-09-18T00:58:54.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589561+0000", 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9207000000000001e-05 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589577+0000", 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5933000000000001e-05 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642111+0000", 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052533799999999999 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642271+0000", 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015998499999999999 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642589+0000", 2024-09-18T00:58:54.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031818999999999999 2024-09-18T00:58:54.383 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.383 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.383 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.383 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64355 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.383 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:54.383 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736446358999999, 2024-09-18T00:58:54.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053570327000000001, 2024-09-18T00:58:54.383 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.383 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64355 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:54.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589599+0000", 2024-09-18T00:58:54.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T00:58:54.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589605+0000", 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8830000000000003e-06 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589622+0000", 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.732e-05 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589636+0000", 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3703000000000001e-05 2024-09-18T00:58:54.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642672+0000", 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053036204000000003 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642828+0000", 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155451 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643168+0000", 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034051900000000001 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64356 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736388846000001, 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054203299000000003, 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.388 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64356 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:54.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589656+0000", 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589662+0000", 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9800000000000003e-06 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589681+0000", 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8300000000000001e-05 2024-09-18T00:58:54.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589695+0000", 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3997e-05 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643249+0000", 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053554101999999999 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643423+0000", 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174062 2024-09-18T00:58:54.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643859+0000", 2024-09-18T00:58:54.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043562499999999998 2024-09-18T00:58:54.392 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.392 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.392 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.392 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64357 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.393 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:54.393 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736330895999998, 2024-09-18T00:58:54.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054239849, 2024-09-18T00:58:54.393 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.393 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.393 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:54.393 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.393 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.393 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64357 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589714+0000", 2024-09-18T00:58:54.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.266e-06 2024-09-18T00:58:54.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589720+0000", 2024-09-18T00:58:54.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2340000000000002e-06 2024-09-18T00:58:54.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589738+0000", 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7932999999999999e-05 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589752+0000", 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4867e-05 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643934+0000", 2024-09-18T00:58:54.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054181721000000002 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643953+0000", 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8828e-05 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64358 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736273016999998, 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054261125, 2024-09-18T00:58:54.397 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64358 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589772+0000", 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1170000000000001e-06 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589778+0000", 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8429999999999998e-06 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589799+0000", 2024-09-18T00:58:54.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0727e-05 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589813+0000", 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4403000000000001e-05 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644021+0000", 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054207784000000002 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644032+0000", 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1251e-05 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64359 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736211051000002, 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286586999999997, 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.401 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.402 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.402 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64359 2024-09-18T00:58:54.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.402 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:54.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589834+0000", 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589841+0000", 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.956e-06 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589864+0000", 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3124000000000001e-05 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589879+0000", 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.486e-05 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644099+0000", 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054219429 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644120+0000", 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1004000000000001e-05 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.405 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64360 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736146857000001, 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054290699999999997, 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64360 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589898+0000", 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3600000000000002e-07 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589903+0000", 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4669999999999998e-06 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589913+0000", 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0463e-05 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589921+0000", 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7959999999999996e-06 2024-09-18T00:58:54.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644173+0000", 2024-09-18T00:58:54.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054251903999999997 2024-09-18T00:58:54.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644188+0000", 2024-09-18T00:58:54.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5133999999999999e-05 2024-09-18T00:58:54.409 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64361 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736113377999999, 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054320101000000003, 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.410 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64361 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5000000000000002e-07 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589935+0000", 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2799999999999999e-06 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589944+0000", 2024-09-18T00:58:54.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589954+0000", 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0036e-05 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644240+0000", 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286273000000003 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644251+0000", 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0392e-05 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64362 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736080231999999, 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054340196, 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.414 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64362 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:54.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589978+0000", 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3570000000000001e-05 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589991+0000", 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3196999999999999e-05 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590000+0000", 2024-09-18T00:58:54.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2800000000000003e-06 2024-09-18T00:58:54.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644294+0000", 2024-09-18T00:58:54.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054294873 2024-09-18T00:58:54.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644304+0000", 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6560000000000004e-06 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64363 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.736034201999999, 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054356660000000001, 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.419 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64363 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590011+0000", 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7400000000000003e-07 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590014+0000", 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:58:54.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590024+0000", 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5670000000000001e-06 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590031+0000", 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3830000000000003e-06 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644354+0000", 2024-09-18T00:58:54.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054323429999999999 2024-09-18T00:58:54.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644367+0000", 2024-09-18T00:58:54.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.217e-05 2024-09-18T00:58:54.423 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.423 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.423 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.423 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.423 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.423 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:54.425 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:54.425 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:54.421+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:54.526 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:54.521+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:54.597 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:54.598 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:54.598 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:54.598 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout:3664 write 0x373e59c thru 0x373f3a8 (0xe0d bytes) 2024-09-18T00:58:54.611 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout:3665 punch from 0x4f96680 to 0x4fa6500, (0xfe80 bytes) 2024-09-18T00:58:54.626 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:54.621+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout:3667 write 0xed67ab2 thru 0xed684c3 (0xa12 bytes) 2024-09-18T00:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout:3668 trunc from 0xed684c4 to 0xda7f6b1 2024-09-18T00:58:54.663 INFO:teuthology.orchestra.run.smithi042.stdout:3669 write 0x2476f57 thru 0x2485601 (0xe6ab bytes) 2024-09-18T00:58:54.701 INFO:teuthology.orchestra.run.smithi042.stdout:3672 write 0xd798269 thru 0xd7a0c37 (0x89cf bytes) 2024-09-18T00:58:54.724 INFO:teuthology.orchestra.run.smithi042.stdout:3673 write 0x7d7514a thru 0x7d75429 (0x2e0 bytes) 2024-09-18T00:58:54.727 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:54.721+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:54.729 INFO:teuthology.orchestra.run.smithi042.stdout:3674 write 0xc0d2a5f thru 0xc0d5b14 (0x30b6 bytes) 2024-09-18T00:58:54.740 INFO:teuthology.orchestra.run.smithi042.stdout:3675 punch from 0xc3d37b7 to 0xc3d6b20, (0x3369 bytes) 2024-09-18T00:58:54.740 INFO:teuthology.orchestra.run.smithi042.stdout:3676 read 0x86e6509 thru 0x86e70f0 (0xbe8 bytes) 2024-09-18T00:58:54.741 INFO:teuthology.orchestra.run.smithi042.stdout:3677 write 0xbf5eff8 thru 0xbf65193 (0x619c bytes) 2024-09-18T00:58:54.741 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:54.742 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:54.756 INFO:teuthology.orchestra.run.smithi042.stdout:3678 read 0x4402ef2 thru 0x440fc27 (0xcd36 bytes) 2024-09-18T00:58:54.756 INFO:teuthology.orchestra.run.smithi042.stdout:3681 read 0x4e9311a thru 0x4e979d7 (0x48be bytes) 2024-09-18T00:58:54.756 INFO:teuthology.orchestra.run.smithi042.stdout:3682 writesame 0x8c63f09 thru 0x8c6f832 data_size 0x2(0xb92a bytes) 2024-09-18T00:58:54.757 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:54.782 INFO:teuthology.orchestra.run.smithi042.stdout:3683 read 0xce903cf thru 0xce9e040 (0xdc72 bytes) 2024-09-18T00:58:54.782 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:58:54.782 INFO:teuthology.orchestra.run.smithi042.stdout:3687 read 0x26057ce thru 0x2609ad1 (0x4304 bytes) 2024-09-18T00:58:54.783 INFO:teuthology.orchestra.run.smithi042.stdout:3688 punch from 0x5b5f706 to 0x5b5fe7a, (0x774 bytes) 2024-09-18T00:58:54.783 INFO:teuthology.orchestra.run.smithi042.stdout:3689 read 0xaa75e34 thru 0xaa814f4 (0xb6c1 bytes) 2024-09-18T00:58:54.806 INFO:teuthology.orchestra.run.smithi042.stdout:3690 trunc from 0xda7f6b1 to 0x19cad28 2024-09-18T00:58:54.849 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:54.845+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:54.924 INFO:teuthology.orchestra.run.smithi042.stderr:marked out osd.2. 2024-09-18T00:58:54.929 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:54.925+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:54.954 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.863411497000001, 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T00:58:54.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T00:58:54.959 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:54.960 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.472516545999994, 2024-09-18T00:58:54.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T00:58:54.960 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.960 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.960 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:54.960 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:54.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:54.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T00:58:54.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T00:58:54.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T00:58:54.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T00:58:54.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T00:58:54.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T00:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78503 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.123792425, 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053463986999999998, 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.965 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78503 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:54.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6899999999999997e-07 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828236+0000", 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.853e-06 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828243+0000", 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4359999999999996e-06 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828257+0000", 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3862999999999999e-05 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881475+0000", 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053217299000000003 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881569+0000", 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4744e-05 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881697+0000", 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012722300000000001 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.969 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78504 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 897024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.123758520999999, 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053679381999999998, 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78504 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828572+0000", 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030532900000000002 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828577+0000", 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8579999999999999e-06 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828596+0000", 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9414000000000001e-05 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828604+0000", 2024-09-18T00:58:54.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0190000000000006e-06 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881742+0000", 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053137864 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881822+0000", 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0346000000000006e-05 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881946+0000", 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123552 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78505 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.123404833, 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053688825000000003, 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.974 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78505 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828656+0000", 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6273999999999999e-05 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828660+0000", 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.76e-06 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828670+0000", 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5279999999999992e-06 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828678+0000", 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8560000000000007e-06 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881994+0000", 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053316364999999998 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882203+0000", 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020851600000000001 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882309+0000", 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106526 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78506 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:54.978 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.123336023, 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032405999999998, 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78506 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:54.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828976+0000", 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028726700000000001 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828983+0000", 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.246e-06 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828996+0000", 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3125999999999999e-05 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829003+0000", 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6899999999999992e-06 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882345+0000", 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053341462999999999 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882423+0000", 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8338999999999997e-05 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882721+0000", 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029827500000000003 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.982 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78507 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.123008762, 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053936698999999998, 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.983 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78507 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829065+0000", 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8863999999999998e-05 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829069+0000", 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9160000000000003e-06 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829080+0000", 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1087000000000001e-05 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829088+0000", 2024-09-18T00:58:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2609999999999993e-06 2024-09-18T00:58:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882766+0000", 2024-09-18T00:58:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053678037999999997 2024-09-18T00:58:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882847+0000", 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0131999999999998e-05 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882953+0000", 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106401 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78508 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 978944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.122925344, 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053997636000000002, 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78508 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830739+0000", 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001639738 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830745+0000", 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8869999999999997e-06 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830755+0000", 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8830000000000001e-06 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830765+0000", 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7589999999999992e-06 2024-09-18T00:58:54.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882990+0000", 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052225252 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.883040+0000", 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9848000000000001e-05 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.883097+0000", 2024-09-18T00:58:54.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7268999999999997e-05 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78514 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1101824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.121117715, 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053776592999999998, 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.992 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78514 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830908+0000", 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830928+0000", 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9898000000000002e-05 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830940+0000", 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2483999999999999e-05 2024-09-18T00:58:54.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830955+0000", 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5143999999999999e-05 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884176+0000", 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053221151000000001 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884581+0000", 2024-09-18T00:58:54.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040494199999999999 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884684+0000", 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102476 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78515 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1122304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.12106, 2024-09-18T00:58:54.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053953972000000003, 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78515 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830966+0000", 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4799999999999998e-07 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830968+0000", 2024-09-18T00:58:54.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6390000000000001e-06 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830975+0000", 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4790000000000001e-06 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830980+0000", 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4269999999999999e-06 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884731+0000", 2024-09-18T00:58:54.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053750345999999997 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884785+0000", 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.49e-05 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884919+0000", 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133533 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.000 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78516 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.121036432, 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054090795999999997, 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78516 2024-09-18T00:58:55.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830991+0000", 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.221e-06 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830998+0000", 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5409999999999997e-06 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831003+0000", 2024-09-18T00:58:55.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2390000000000002e-06 2024-09-18T00:58:55.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884953+0000", 2024-09-18T00:58:55.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053950117999999998 2024-09-18T00:58:55.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885000+0000", 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6555999999999999e-05 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885079+0000", 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9702000000000004e-05 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78517 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1163264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:55.005 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.121015496, 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298990999999998, 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78517 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4400000000000001e-07 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831018+0000", 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2490000000000001e-06 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831025+0000", 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831031+0000", 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.463e-06 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885130+0000", 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054099371 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885207+0000", 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7292000000000005e-05 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885309+0000", 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010129899999999999 2024-09-18T00:58:55.009 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78518 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1183744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.120985717, 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054532499999999998, 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78518 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831040+0000", 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3099999999999997e-07 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831042+0000", 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2879999999999999e-06 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831050+0000", 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.464e-06 2024-09-18T00:58:55.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831055+0000", 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0819999999999998e-06 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885354+0000", 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298739999999998 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885428+0000", 2024-09-18T00:58:55.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4567999999999995e-05 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885572+0000", 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014372699999999999 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78519 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1204224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.120963868, 2024-09-18T00:58:55.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054707261, 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78519 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831062+0000", 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831064+0000", 2024-09-18T00:58:55.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0360000000000001e-06 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831069+0000", 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4149999999999998e-06 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831075+0000", 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2249999999999997e-06 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885602+0000", 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054526893 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885673+0000", 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0865000000000007e-05 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885768+0000", 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5422999999999998e-05 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78520 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1224704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.120943412000001, 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054906695999999998, 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78520 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.019 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:55.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831084+0000", 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2960000000000001e-06 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831091+0000", 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831095+0000", 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.792e-06 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885802+0000", 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054706795000000003 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885884+0000", 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2297000000000002e-05 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885988+0000", 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010393199999999999 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78521 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.120898539000001, 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055066616999999998, 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78521 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831127+0000", 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.99e-07 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831130+0000", 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6819999999999999e-06 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831137+0000", 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3849999999999996e-06 2024-09-18T00:58:55.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831145+0000", 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7360000000000001e-06 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886020+0000", 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054874700999999998 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886096+0000", 2024-09-18T00:58:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6234000000000002e-05 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886193+0000", 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7280000000000004e-05 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78522 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.120872778000001, 2024-09-18T00:58:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055137938999999997, 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78522 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831153+0000", 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.37e-07 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831155+0000", 2024-09-18T00:58:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4420000000000001e-06 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831161+0000", 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831166+0000", 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8749999999999999e-06 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886279+0000", 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055112337999999997 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886290+0000", 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1527e-05 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78523 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.120852098, 2024-09-18T00:58:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055171425000000003, 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78523 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0900000000000002e-07 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831176+0000", 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4830000000000002e-06 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831183+0000", 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0679999999999999e-06 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831189+0000", 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886331+0000", 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055142669999999998 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886344+0000", 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2918999999999999e-05 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78524 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.120829982, 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055194430000000003, 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78524 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831196+0000", 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1200000000000003e-07 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831198+0000", 2024-09-18T00:58:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831204+0000", 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2740000000000004e-06 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831209+0000", 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0900000000000004e-06 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886381+0000", 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055172142 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886389+0000", 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2150000000000001e-06 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78525 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.120809468999999, 2024-09-18T00:58:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055206535000000001, 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78525 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832358+0000", 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001142531 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832364+0000", 2024-09-18T00:58:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832375+0000", 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1381e-05 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832382+0000", 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9800000000000001e-06 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886415+0000", 2024-09-18T00:58:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032678000000001 2024-09-18T00:58:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886422+0000", 2024-09-18T00:58:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4449999999999999e-06 2024-09-18T00:58:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.043 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:55.045 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:55.045 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:55.037+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:55.146 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:55.141+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:55.215 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:55.216 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:55.216 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:55.216 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:55.230 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:55.246 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:55.241+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:55.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:55.347 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:55.341+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:55.352 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:55.352 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:55.352 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:55.352 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:55.352 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:55.366 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:55.438 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:55.438 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:55.448 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:55.441+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:55.454 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:55.455 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:55.471 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4041 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.481060607000003, 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037088000000000003, 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.487 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4041 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003005+0000", 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.187e-06 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003011+0000", 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003029+0000", 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8026999999999999e-05 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003043+0000", 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039484+0000", 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036440895000000001 2024-09-18T00:58:55.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039722+0000", 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000238507 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040092+0000", 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000369818 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.491 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.481002984, 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:58:55.495 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.496 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.496 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.496 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:55.496 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:58:55.496 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.480944726999994, 2024-09-18T00:58:55.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:58:55.496 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.496 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.497 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.497 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.497 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.497 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.497 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:58:55.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.497 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:58:55.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:58:55.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.480886287999994, 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:58:55.501 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:58:55.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.480825070999998, 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:58:55.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.480734655999996, 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.510 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.514 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.480663723000006, 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:58:55.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.480604116999999, 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.519 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.480544746999996, 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.523 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:58:55.527 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.480482957999996, 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:58:55.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.531 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.480350134999995, 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:58:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 66.925132657999995, 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:55.536 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:58:55.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T00:58:55.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.541 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 66.924632533999997, 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T00:58:55.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T00:58:55.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 66.924602026000002, 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.546 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T00:58:55.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 66.924577361000004, 2024-09-18T00:58:55.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T00:58:55.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 66.924548829000003, 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T00:58:55.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 66.924215537999999, 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.559 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:58:55.563 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 66.924173687000007, 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T00:58:55.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T00:58:55.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:55.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:58:55.569 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 66.923136190999998, 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T00:58:55.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.270848185999998, 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T00:58:55.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:55.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T00:58:55.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T00:58:55.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.579 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.580 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:55.580 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:55.580 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:55.581 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:55.581 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:55.582 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:55.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:55.583 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:58:55.583 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:55.583 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:55.584 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:55.584 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:55.585 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:55.573+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:55.682 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:55.681+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:55.708 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:55.708 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:55.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:58:55.782 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:55.781+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:55.842 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:55.842 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:55.857 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:58:55.902 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:55.897+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:55.946 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:58:55.982 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:55.983 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:55.985 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:55.985+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:55.998 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:58:56.086 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:56.081+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:56.187 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:56.181+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:56.209 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:56.209 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:56.224 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:58:56.310 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:56.305+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:56.382 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:56.382 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:56.388 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:56.381+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:56.395 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:58:56.489 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:56.481+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:56.561 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:56.562 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:56.576 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:58:56.618 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:56.613+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:56.693 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:56.685+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:56.740 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:56.740 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:56.755 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:58:56.793 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:56.793+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:56.894 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:56.889+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:56.928 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:58:56.932 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:56.932 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:56.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:58:56.988 INFO:teuthology.orchestra.run.smithi042.stdout:3691 write 0x4f16ace thru 0x4f17922 (0xe55 bytes) 2024-09-18T00:58:56.994 INFO:teuthology.orchestra.run.smithi042.stdout:3692 punch from 0x139bce2 to 0x13a7f95, (0xc2b3 bytes) 2024-09-18T00:58:56.994 INFO:teuthology.orchestra.run.smithi042.stdout:3694 read 0x34a29a8 thru 0x34ad10d (0xa766 bytes) 2024-09-18T00:58:56.994 INFO:teuthology.orchestra.run.smithi042.stdout:3695 punch from 0x31c4373 to 0x31c55ff, (0x128c bytes) 2024-09-18T00:58:56.995 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:56.989+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:57.009 INFO:teuthology.orchestra.run.smithi042.stdout:3696 writesame 0xc2754c9 thru 0xc282ebd data_size 0x7(0xd9f5 bytes) 2024-09-18T00:58:57.016 INFO:teuthology.orchestra.run.smithi042.stdout:3700 write 0xcc5f90d thru 0xcc670a2 (0x7796 bytes) 2024-09-18T00:58:57.033 INFO:teuthology.orchestra.run.smithi042.stdout:3702 writesame 0x190328a thru 0x190d2f3 data_size 0x1(0xa06a bytes) 2024-09-18T00:58:57.048 INFO:teuthology.orchestra.run.smithi042.stdout:3703 read 0x608df2c thru 0x608ec0e (0xce3 bytes) 2024-09-18T00:58:57.048 INFO:teuthology.orchestra.run.smithi042.stdout:3704 writesame 0xc719f4f thru 0xc71daca data_size 0x36(0x3b7c bytes) 2024-09-18T00:58:57.056 INFO:teuthology.orchestra.run.smithi042.stdout:3706 clone 30 order 21 su 16384 sc 5 2024-09-18T00:58:57.096 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:57.089+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:57.107 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:57.107 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:57.124 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:58:57.197 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:57.193+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:57.287 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:57.288 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:57.298 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:57.293+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:57.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:58:57.399 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:57.393+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:57.430 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:57.430 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:57.445 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:58:57.534 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:57.529+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:57.594 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:58:57.600 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:57.593+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:57.629 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:57.629 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:57.646 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:58:57.701 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:58:57.697+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:58:57.759 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:58:57.760 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:58:57.777 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:58:57.777 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:58:57.791 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:58:57.791 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:58:57.796 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:58:57.796 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:58:57.803 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:57.797+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:57.806 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:58:57.818 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:58:57.818 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:58:57.839 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:58:57.839 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:58:57.857 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:58:57.858 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:58:57.879 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:58:57.879 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:58:57.900 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:58:57.900 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:58:57.904 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:57.897+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:57.987 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:57.987 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:57.987 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:57.987 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:57.987 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:58.000 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:58:58.045 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:58.041+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:58.106 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:58.105+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:58.173 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:58.173 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:58.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:58.173 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:58.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.173 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1672 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.173 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:58.173 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.003123388000006, 2024-09-18T00:58:58.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083853480999999994, 2024-09-18T00:58:58.173 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1672 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166696+0000", 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166697+0000", 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0920000000000001e-06 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166702+0000", 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0799999999999996e-06 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166718+0000", 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6167999999999998e-05 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166745+0000", 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7177000000000001e-05 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250382+0000", 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083636792000000001 2024-09-18T00:58:58.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250476+0000", 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250549+0000", 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3282999999999996e-05 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1673 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.003056279999996, 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084151880999999998, 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1673 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.178 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166763+0000", 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166764+0000", 2024-09-18T00:58:58.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166770+0000", 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5969999999999999e-06 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166786+0000", 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6439999999999998e-05 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166799+0000", 2024-09-18T00:58:58.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2934e-05 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250605+0000", 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083806357999999997 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250724+0000", 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011821500000000001 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250915+0000", 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000191251 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1674 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.002956694999995, 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084239101999999996, 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.182 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1674 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166863+0000", 2024-09-18T00:58:58.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166864+0000", 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3629999999999999e-06 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166883+0000", 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9083e-05 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166902+0000", 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8711000000000001e-05 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166915+0000", 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3375e-05 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.250962+0000", 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084047254000000002 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251029+0000", 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6315000000000005e-05 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251102+0000", 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3001000000000001e-05 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.186 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1675 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1540096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.002884843999993, 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084536891000000003, 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1675 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166934+0000", 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166936+0000", 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2720000000000001e-06 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166941+0000", 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7370000000000003e-06 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166958+0000", 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166972+0000", 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3569e-05 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251159+0000", 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084187467000000002 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251300+0000", 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014059200000000001 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251471+0000", 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017168000000000001 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.190 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1676 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1581056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.002831963999995, 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084738637000000006, 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.191 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1676 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166987+0000", 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166988+0000", 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.166993+0000", 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8339999999999998e-06 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167009+0000", 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6028e-05 2024-09-18T00:58:58.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167022+0000", 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2675e-05 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251517+0000", 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084494975999999999 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251576+0000", 2024-09-18T00:58:58.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8644999999999998e-05 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251726+0000", 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015039 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1677 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.002780293000001, 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085055403000000002, 2024-09-18T00:58:58.195 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1677 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167039+0000", 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167040+0000", 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167046+0000", 2024-09-18T00:58:58.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2024-09-18T00:58:58.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167062+0000", 2024-09-18T00:58:58.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6354e-05 2024-09-18T00:58:58.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167075+0000", 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2965e-05 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251789+0000", 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084713940000000001 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.251916+0000", 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000127176 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252094+0000", 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017833599999999999 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1678 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1662976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.002728529999999, 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085316746999999998, 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.200 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1678 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167091+0000", 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167092+0000", 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167097+0000", 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7029999999999997e-06 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167113+0000", 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167125+0000", 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2517000000000001e-05 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252157+0000", 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085031933000000004 2024-09-18T00:58:58.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252277+0000", 2024-09-18T00:58:58.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119708 2024-09-18T00:58:58.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252407+0000", 2024-09-18T00:58:58.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013031800000000001 2024-09-18T00:58:58.204 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1679 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.002677810999998, 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085557195000000003, 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1679 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167141+0000", 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167143+0000", 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167170+0000", 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7523000000000001e-05 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167189+0000", 2024-09-18T00:58:58.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8530999999999999e-05 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167215+0000", 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6676999999999999e-05 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252445+0000", 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085229416000000002 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252529+0000", 2024-09-18T00:58:58.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4740000000000005e-05 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252699+0000", 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016920800000000001 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1680 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:58.209 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.002584354000007, 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085810110999999994, 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1680 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:58.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167235+0000", 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167236+0000", 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.048e-06 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167242+0000", 2024-09-18T00:58:58.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6459999999999998e-06 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167259+0000", 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.696e-05 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167271+0000", 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.292e-05 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252839+0000", 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085567292000000003 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.252924+0000", 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5511000000000002e-05 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253045+0000", 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120734 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.213 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1681 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.002478257999996, 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085887343000000005, 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1681 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.214 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167341+0000", 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167342+0000", 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167348+0000", 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4519999999999996e-06 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167372+0000", 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4261999999999999e-05 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167400+0000", 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253091+0000", 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085690895000000003 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253154+0000", 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3262000000000005e-05 2024-09-18T00:58:58.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253228+0000", 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4318000000000002e-05 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1682 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.002400178000002, 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.086061112999999995, 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.218 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1682 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.167419+0000", 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168114+0000", 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00069532600000000002 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168123+0000", 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8319999999999995e-06 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168147+0000", 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3536999999999998e-05 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168163+0000", 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6317e-05 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253274+0000", 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085111390999999995 2024-09-18T00:58:58.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253362+0000", 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7384000000000002e-05 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253480+0000", 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011832600000000001 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1683 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1867776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.001623046999995, 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085456638000000001, 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1683 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.223 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.168196+0000", 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169938+0000", 2024-09-18T00:58:58.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017419079999999999 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169951+0000", 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2534999999999999e-05 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.169990+0000", 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9004999999999997e-05 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170011+0000", 2024-09-18T00:58:58.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.156e-05 2024-09-18T00:58:58.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253534+0000", 2024-09-18T00:58:58.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083522759000000002 2024-09-18T00:58:58.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253585+0000", 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1316999999999997e-05 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253653+0000", 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7553999999999997e-05 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1685 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.999712614000003, 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083900785000000005, 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1685 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170107+0000", 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170108+0000", 2024-09-18T00:58:58.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.229e-06 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170114+0000", 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0079999999999999e-06 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170132+0000", 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7734e-05 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170145+0000", 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2977e-05 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253866+0000", 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083721208000000005 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.253934+0000", 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.834e-05 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254007+0000", 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3288999999999996e-05 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1686 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.999657403, 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084083526000000006, 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.232 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1686 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170162+0000", 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170163+0000", 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1909999999999999e-06 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170180+0000", 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7323e-05 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170198+0000", 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7723e-05 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170212+0000", 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3645999999999999e-05 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254050+0000", 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083838115000000005 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254136+0000", 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6106999999999994e-05 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254245+0000", 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010942099999999999 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.236 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1687 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.999589100999998, 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084186574, 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1687 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170230+0000", 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170647+0000", 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416649 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170655+0000", 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6510000000000007e-06 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170683+0000", 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7093999999999998e-05 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170701+0000", 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8383e-05 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254283+0000", 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083582091999999997 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254339+0000", 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6187999999999998e-05 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254417+0000", 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7516999999999997e-05 2024-09-18T00:58:58.240 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:1688 3.19 3:9c923d2e:::rbd_data.117c6d19cb58.0000000000000003:3 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.999093387000002, 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.083931026000000006, 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.241 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1688 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.170726+0000", 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173553+0000", 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002827333 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173566+0000", 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3276e-05 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173603+0000", 2024-09-18T00:58:58.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6177000000000003e-05 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.173628+0000", 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5343000000000001e-05 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254460+0000", 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.080831745999999996 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254547+0000", 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7045000000000004e-05 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:44.254657+0000", 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011010600000000001 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.394956184999998, 2024-09-18T00:58:58.245 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:58:58.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87406 3.19 3:9d734bda:::rbd_data.117c8934add3.000000000000007c:head [stat,delete] snapc 1f=[1f] RETRY=8 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.7470330629999999, 2024-09-18T00:58:58.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097024104999999999, 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87406 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425532+0000", 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027458309999999998 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425537+0000", 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2839999999999997e-06 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425551+0000", 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3682999999999999e-05 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425559+0000", 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5329999999999997e-06 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515035+0000", 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.089476411000000006 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515168+0000", 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013309699999999999 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515475+0000", 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030662199999999999 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519553+0000", 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040781259999999996 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519665+0000", 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112669 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519748+0000", 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2270999999999999e-05 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519810+0000", 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2577999999999996e-05 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87408 3.19 3:9f063cdd:::rbd_data.117c8934add3.000000000000007d:head [stat,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425586+0000", 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.7442329599999997, 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.094380550999999993, 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.257 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87408 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425586+0000", 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425586+0000", 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425587+0000", 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8599999999999998e-07 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425590+0000", 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6790000000000001e-06 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425596+0000", 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3690000000000003e-06 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425600+0000", 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5549999999999996e-06 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.516522+0000", 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.090921960999999996 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.516767+0000", 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000244202 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.517317+0000", 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055067199999999999 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519642+0000", 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023243790000000001 2024-09-18T00:58:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519827+0000", 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184816 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519909+0000", 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2799000000000005e-05 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519967+0000", 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7533e-05 2024-09-18T00:58:58.262 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87414 3.19 3:9c145b25:::rbd_data.117c8934add3.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.7442095240000004, 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095886693999999995, 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.263 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87414 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425618+0000", 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9810000000000003e-06 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425625+0000", 2024-09-18T00:58:58.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0870000000000001e-06 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425630+0000", 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.262e-06 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518417+0000", 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092787816999999995 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518593+0000", 2024-09-18T00:58:58.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017569100000000001 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518939+0000", 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034572299999999998 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521320+0000", 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023809619999999999 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:58.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521343+0000", 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3317000000000001e-05 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521422+0000", 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480999999999996e-05 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521496+0000", 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4909999999999999e-05 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.268 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.269 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.269 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:58.270 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:58:58.270 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:58.265+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:58.371 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:58.365+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:58.441 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:58.441 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:58.441 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:58.441 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:58.455 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:58:58.501 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:58.497+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:58.573 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:58.569+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:58.578 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:58.578 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:58.578 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:58.578 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:58.578 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:58.592 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:58:58.673 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:58.665+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:58.766 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:58.766 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:58.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:58.766 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:58.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.766 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:58:58.766 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:58.766 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.354249320999998, 2024-09-18T00:58:58.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T00:58:58.766 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.766 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T00:58:58.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T00:58:58.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:26797 3.0 3:03a9420d:::rbd_data.117c4c7db1a0.0000000000000071:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.575822522000003, 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073789632999999993, 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.772 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 26797 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187640+0000", 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187653+0000", 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2357e-05 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187684+0000", 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1582999999999999e-05 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187699+0000", 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5003e-05 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187806+0000", 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106776 2024-09-18T00:58:58.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.188026+0000", 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021975799999999999 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7", 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.201663+0000", 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013637768 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261226+0000", 2024-09-18T00:58:58.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059562916 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261261+0000", 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4360000000000003e-05 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261354+0000", 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3041999999999995e-05 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261427+0000", 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3553000000000005e-05 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64346 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.174570936000002, 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048905196999999997, 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.778 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64346 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:58:58.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588890+0000", 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3e-07 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588893+0000", 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.106e-06 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588904+0000", 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1037e-05 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588911+0000", 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2729999999999997e-06 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637426+0000", 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048514483999999997 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637537+0000", 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110901 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637795+0000", 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025776599999999998 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64347 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:58.782 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.174539062999997, 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049469502999999998, 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64347 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:58.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588922+0000", 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4300000000000003e-07 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588925+0000", 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588934+0000", 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0329999999999997e-06 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588946+0000", 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1547e-05 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637872+0000", 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048926202000000002 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638047+0000", 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174566 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638391+0000", 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034401499999999998 2024-09-18T00:58:58.786 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64348 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.174498192999998, 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050239487999999999, 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.787 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64348 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588963+0000", 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.133e-06 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588969+0000", 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4229999999999996e-06 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588996+0000", 2024-09-18T00:58:58.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7396e-05 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589163+0000", 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016677900000000001 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638498+0000", 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049334743 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638715+0000", 2024-09-18T00:58:58.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000217414 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639202+0000", 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048660000000000001 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64349 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.174269287999998, 2024-09-18T00:58:58.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050607656000000001, 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64349 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589192+0000", 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.536e-06 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589201+0000", 2024-09-18T00:58:58.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-06 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589222+0000", 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589241+0000", 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8960000000000001e-05 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639285+0000", 2024-09-18T00:58:58.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050043892999999999 2024-09-18T00:58:58.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639457+0000", 2024-09-18T00:58:58.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017286299999999999 2024-09-18T00:58:58.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639799+0000", 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034117400000000002 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64350 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.174198943, 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051028502000000003, 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.796 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64350 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:58:58.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589262+0000", 2024-09-18T00:58:58.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1570000000000001e-06 2024-09-18T00:58:58.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589280+0000", 2024-09-18T00:58:58.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7943000000000001e-05 2024-09-18T00:58:58.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589299+0000", 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8940000000000002e-05 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589316+0000", 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6654e-05 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639872+0000", 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050556151000000001 2024-09-18T00:58:58.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639988+0000", 2024-09-18T00:58:58.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115423 2024-09-18T00:58:58.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640290+0000", 2024-09-18T00:58:58.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030223400000000002 2024-09-18T00:58:58.800 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.800 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.801 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.801 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64351 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.801 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:58.801 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.174123203000001, 2024-09-18T00:58:58.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051389975999999997, 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64351 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589338+0000", 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589345+0000", 2024-09-18T00:58:58.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589363+0000", 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8346e-05 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589381+0000", 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7357e-05 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640373+0000", 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050992347 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640504+0000", 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000131011 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640727+0000", 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022310799999999999 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64352 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.174059673999999, 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051932649999999997, 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64352 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.806 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589402+0000", 2024-09-18T00:58:58.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.237e-06 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589408+0000", 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2770000000000002e-06 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589426+0000", 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8006e-05 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589441+0000", 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4934e-05 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640803+0000", 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051361619999999997 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640986+0000", 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018303199999999999 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641333+0000", 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034754399999999998 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64353 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.174001279999999, 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052565148999999999, 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.810 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64353 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589460+0000", 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4300000000000001e-06 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589466+0000", 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.446e-06 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589485+0000", 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8726999999999999e-05 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589499+0000", 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.402e-05 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641427+0000", 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051927945000000003 2024-09-18T00:58:58.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641601+0000", 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174859 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642024+0000", 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000422722 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64354 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.173926581000003, 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053055607999999997, 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64354 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589535+0000", 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3430000000000001e-06 2024-09-18T00:58:58.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589542+0000", 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589561+0000", 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9207000000000001e-05 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589577+0000", 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5933000000000001e-05 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642111+0000", 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052533799999999999 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642271+0000", 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015998499999999999 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642589+0000", 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031818999999999999 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64355 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.173862544999999, 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053570327000000001, 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.819 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64355 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:58:58.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589599+0000", 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589605+0000", 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8830000000000003e-06 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589622+0000", 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.732e-05 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589636+0000", 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3703000000000001e-05 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642672+0000", 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053036204000000003 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642828+0000", 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155451 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643168+0000", 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034051900000000001 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64356 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.823 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.173805031999997, 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054203299000000003, 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64356 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589656+0000", 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589662+0000", 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9800000000000003e-06 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589681+0000", 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8300000000000001e-05 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589695+0000", 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3997e-05 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643249+0000", 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053554101999999999 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643423+0000", 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174062 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643859+0000", 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043562499999999998 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.827 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.828 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64357 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.828 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:58.828 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.173747081999998, 2024-09-18T00:58:58.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054239849, 2024-09-18T00:58:58.828 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.828 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.828 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:58.828 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.829 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.829 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.829 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64357 2024-09-18T00:58:58.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.829 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:58.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589714+0000", 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.266e-06 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589720+0000", 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2340000000000002e-06 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589738+0000", 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7932999999999999e-05 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589752+0000", 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4867e-05 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643934+0000", 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054181721000000002 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643953+0000", 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8828e-05 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.832 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64358 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.173689203000002, 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054261125, 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64358 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589772+0000", 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1170000000000001e-06 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589778+0000", 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8429999999999998e-06 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589799+0000", 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0727e-05 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589813+0000", 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4403000000000001e-05 2024-09-18T00:58:58.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644021+0000", 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054207784000000002 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644032+0000", 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1251e-05 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64359 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.836 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.173627236999998, 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286586999999997, 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64359 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589834+0000", 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589841+0000", 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.956e-06 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589864+0000", 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3124000000000001e-05 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589879+0000", 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.486e-05 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644099+0000", 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054219429 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644120+0000", 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1004000000000001e-05 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.840 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64360 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.173563043000001, 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054290699999999997, 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64360 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589898+0000", 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3600000000000002e-07 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589903+0000", 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4669999999999998e-06 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589913+0000", 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0463e-05 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589921+0000", 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7959999999999996e-06 2024-09-18T00:58:58.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644173+0000", 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054251903999999997 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644188+0000", 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5133999999999999e-05 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64361 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.173529563999999, 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054320101000000003, 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64361 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5000000000000002e-07 2024-09-18T00:58:58.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589935+0000", 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2799999999999999e-06 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589944+0000", 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589954+0000", 2024-09-18T00:58:58.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0036e-05 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644240+0000", 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286273000000003 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644251+0000", 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0392e-05 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64362 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.173496417999999, 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054340196, 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64362 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.849 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:58.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:58.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589978+0000", 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3570000000000001e-05 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589991+0000", 2024-09-18T00:58:58.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3196999999999999e-05 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590000+0000", 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2800000000000003e-06 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644294+0000", 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054294873 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644304+0000", 2024-09-18T00:58:58.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6560000000000004e-06 2024-09-18T00:58:58.853 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.853 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.853 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.853 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64363 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:58:58.853 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:58.853 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.173450387999999, 2024-09-18T00:58:58.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054356660000000001, 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64363 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590011+0000", 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7400000000000003e-07 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590014+0000", 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:58:58.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590024+0000", 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5670000000000001e-06 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590031+0000", 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3830000000000003e-06 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:58.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644354+0000", 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054323429999999999 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644367+0000", 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.217e-05 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:58.857 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:58.859 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:58:58.859 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:58.857+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:58.960 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:58:58.953+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:58:58.970 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:58:58.993 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:58.993 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:58.993 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:58.993 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:59.008 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:58:59.060 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:58:59.053+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:58:59.091 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone29 intact 2024-09-18T00:58:59.161 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:58:59.157+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:58:59.175 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:59.175 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:59.175 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:59.175 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:59.176 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:59.190 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:58:59.262 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:59.257+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:59.322 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-18T00:58:59.363 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:59.357+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:59.365 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:59.365 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:59.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:59.365 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:59.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.365 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T00:58:59.365 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:59.365 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.271538860999996, 2024-09-18T00:58:59.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T00:58:59.365 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.365 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T00:58:59.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.880643910000003, 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T00:58:59.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78503 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.531919789, 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053463986999999998, 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78503 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828233+0000", 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6899999999999997e-07 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828236+0000", 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.853e-06 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828243+0000", 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4359999999999996e-06 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828257+0000", 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3862999999999999e-05 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881475+0000", 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053217299000000003 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881569+0000", 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4744e-05 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881697+0000", 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012722300000000001 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.378 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78504 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 897024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.531885884999999, 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053679381999999998, 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78504 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828267+0000", 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828572+0000", 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030532900000000002 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828577+0000", 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8579999999999999e-06 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828596+0000", 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9414000000000001e-05 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828604+0000", 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0190000000000006e-06 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881742+0000", 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053137864 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881822+0000", 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0346000000000006e-05 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881946+0000", 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123552 2024-09-18T00:58:59.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78505 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.531532197000001, 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053688825000000003, 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.383 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78505 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828620+0000", 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828656+0000", 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6273999999999999e-05 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828660+0000", 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.76e-06 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828670+0000", 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5279999999999992e-06 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828678+0000", 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8560000000000007e-06 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.881994+0000", 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053316364999999998 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882203+0000", 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020851600000000001 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882309+0000", 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106526 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78506 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 937984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.531463387000001, 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032405999999998, 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.387 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78506 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828689+0000", 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828976+0000", 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028726700000000001 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828983+0000", 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.246e-06 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.828996+0000", 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3125999999999999e-05 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829003+0000", 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6899999999999992e-06 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882345+0000", 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053341462999999999 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882423+0000", 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8338999999999997e-05 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882721+0000", 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029827500000000003 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78507 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.531136126, 2024-09-18T00:58:59.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053936698999999998, 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78507 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829016+0000", 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829065+0000", 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8863999999999998e-05 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829069+0000", 2024-09-18T00:58:59.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9160000000000003e-06 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829080+0000", 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1087000000000001e-05 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829088+0000", 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2609999999999993e-06 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882766+0000", 2024-09-18T00:58:59.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053678037999999997 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882847+0000", 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0131999999999998e-05 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882953+0000", 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106401 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78508 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 978944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.531052708000001, 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053997636000000002, 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78508 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.829100+0000", 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830739+0000", 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001639738 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830745+0000", 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8869999999999997e-06 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830755+0000", 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8830000000000001e-06 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830765+0000", 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7589999999999992e-06 2024-09-18T00:58:59.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.882990+0000", 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052225252 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.883040+0000", 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9848000000000001e-05 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.883097+0000", 2024-09-18T00:58:59.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7268999999999997e-05 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78514 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1101824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.529245079000001, 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053776592999999998, 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.400 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78514 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830907+0000", 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830908+0000", 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830928+0000", 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9898000000000002e-05 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830940+0000", 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2483999999999999e-05 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830955+0000", 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5143999999999999e-05 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884176+0000", 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053221151000000001 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884581+0000", 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040494199999999999 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884684+0000", 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102476 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78515 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1122304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.529187364, 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053953972000000003, 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.404 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78515 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830965+0000", 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830966+0000", 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4799999999999998e-07 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830968+0000", 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6390000000000001e-06 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830975+0000", 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4790000000000001e-06 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830980+0000", 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4269999999999999e-06 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884731+0000", 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053750345999999997 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884785+0000", 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.49e-05 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884919+0000", 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133533 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78516 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.408 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.529163796000001, 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054090795999999997, 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78516 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830989+0000", 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830991+0000", 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.221e-06 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.830998+0000", 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5409999999999997e-06 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831003+0000", 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2390000000000002e-06 2024-09-18T00:58:59.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.884953+0000", 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053950117999999998 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885000+0000", 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6555999999999999e-05 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885079+0000", 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9702000000000004e-05 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78517 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1163264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.52914286, 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298990999999998, 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.413 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78517 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831010+0000", 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4400000000000001e-07 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831018+0000", 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2490000000000001e-06 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831025+0000", 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831031+0000", 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.463e-06 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885130+0000", 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054099371 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885207+0000", 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7292000000000005e-05 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885309+0000", 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010129899999999999 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78518 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1183744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.529113081, 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054532499999999998, 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.417 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78518 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831040+0000", 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3099999999999997e-07 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831042+0000", 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2879999999999999e-06 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831050+0000", 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.464e-06 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831055+0000", 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0819999999999998e-06 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885354+0000", 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298739999999998 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885428+0000", 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4567999999999995e-05 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885572+0000", 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014372699999999999 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78519 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1204224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.529091232000001, 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054707261, 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.421 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78519 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831062+0000", 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831064+0000", 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0360000000000001e-06 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831069+0000", 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4149999999999998e-06 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831075+0000", 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2249999999999997e-06 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885602+0000", 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054526893 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885673+0000", 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0865000000000007e-05 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885768+0000", 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5422999999999998e-05 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78520 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1224704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.529070775999999, 2024-09-18T00:58:59.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054906695999999998, 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78520 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831084+0000", 2024-09-18T00:58:59.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2960000000000001e-06 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831091+0000", 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831095+0000", 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.792e-06 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885802+0000", 2024-09-18T00:58:59.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054706795000000003 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885884+0000", 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2297000000000002e-05 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885988+0000", 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010393199999999999 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78521 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.529025903000001, 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055066616999999998, 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78521 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:59.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:58:59.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831127+0000", 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.99e-07 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831130+0000", 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6819999999999999e-06 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831137+0000", 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3849999999999996e-06 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831145+0000", 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7360000000000001e-06 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886020+0000", 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054874700999999998 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886096+0000", 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6234000000000002e-05 2024-09-18T00:58:59.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886193+0000", 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7280000000000004e-05 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78522 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.529000141999999, 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055137938999999997, 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.434 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78522 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:58:59.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831153+0000", 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.37e-07 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831155+0000", 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4420000000000001e-06 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831161+0000", 2024-09-18T00:58:59.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831166+0000", 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8749999999999999e-06 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886279+0000", 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055112337999999997 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886290+0000", 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1527e-05 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78523 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.528979462000001, 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055171425000000003, 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.438 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78523 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0900000000000002e-07 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831176+0000", 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4830000000000002e-06 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831183+0000", 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0679999999999999e-06 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831189+0000", 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886331+0000", 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055142669999999998 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886344+0000", 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2918999999999999e-05 2024-09-18T00:58:59.441 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78524 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.528957346, 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055194430000000003, 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.442 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78524 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831196+0000", 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1200000000000003e-07 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831198+0000", 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831204+0000", 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2740000000000004e-06 2024-09-18T00:58:59.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831209+0000", 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0900000000000004e-06 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886381+0000", 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055172142 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886389+0000", 2024-09-18T00:58:59.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2150000000000001e-06 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78525 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.528936832999999, 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055206535000000001, 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.446 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78525 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832358+0000", 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001142531 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832364+0000", 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832375+0000", 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1381e-05 2024-09-18T00:58:59.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832382+0000", 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9800000000000001e-06 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886415+0000", 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032678000000001 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886422+0000", 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4449999999999999e-06 2024-09-18T00:58:59.449 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.450 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.450 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.450 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.450 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.450 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:59.451 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:58:59.464 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:59.457+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:59.565 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:58:59.561+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:58:59.594 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:58:59.594 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:59.609 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:58:59.665 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:58:59.657+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:58:59.766 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:58:59.761+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:58:59.794 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:59.795 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:58:59.795 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:58:59.795 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:58:59.795 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:58:59.812 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:58:59.867 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:58:59.861+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:58:59.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:58:59.947 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:58:59.947 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:58:59.947 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 3, 4, 5, 6, 7] 2024-09-18T00:58:59.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-18T00:58:59.948 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:58:59.948 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4041 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.942783789999993, 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037088000000000003, 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.949 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4041 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003005+0000", 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.187e-06 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003011+0000", 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003029+0000", 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8026999999999999e-05 2024-09-18T00:58:59.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003043+0000", 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039484+0000", 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036440895000000001 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039722+0000", 2024-09-18T00:58:59.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000238507 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040092+0000", 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000369818 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.942726167000004, 2024-09-18T00:58:59.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.957 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.942667909999997, 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:58:59.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:58:59.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.942609470999997, 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.962 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:58:59.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:58:59.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:58:59.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:58:59.968 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.968 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.968 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.968 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:59.968 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:58:59.968 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.942548254000002, 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:58:59.972 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.942457838999999, 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.973 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:58:59.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:58:59.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.942386905999996, 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:58:59.977 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:58:59.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:58:59.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.942327300000002, 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:58:59.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:58:59.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.94226793, 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.986 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:58:59.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:58:59.990 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.942206141, 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:58:59.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:58:59.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:58:59.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.942073317999998, 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:58:59.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:58:59.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:58:59.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:58:59.999 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 71.386855840999999, 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.000 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:00.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T00:59:00.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:00.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T00:59:00.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:00.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 71.386355717000001, 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:00.006 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:00.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:00.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:00.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T00:59:00.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T00:59:00.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:00.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T00:59:00.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T00:59:00.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:00.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T00:59:00.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T00:59:00.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:00.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T00:59:00.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T00:59:00.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:00.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T00:59:00.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T00:59:00.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:00.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T00:59:00.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T00:59:00.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:00.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T00:59:00.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T00:59:00.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.011 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:00.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 71.386325209000006, 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:00.012 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T00:59:00.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T00:59:00.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 71.386300543999994, 2024-09-18T00:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T00:59:00.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T00:59:00.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 71.386272012000006, 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T00:59:00.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 71.385938721000002, 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:00.025 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T00:59:00.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T00:59:00.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 71.385896869999996, 2024-09-18T00:59:00.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T00:59:00.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:00.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T00:59:00.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T00:59:00.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:00.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T00:59:00.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T00:59:00.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T00:59:00.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 71.384859374000001, 2024-09-18T00:59:00.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:00.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:00.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:00.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.732571368999999, 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T00:59:00.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T00:59:00.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T00:59:00.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:00.045 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:00.046 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:00.048 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:59:00.048 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:00.041+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:00.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:00.093 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:00.115 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:00.116 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:00.136 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:00.148 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:00.141+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:00.153 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:00.173 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:00.194 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:00.194 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:00.214 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:00.214 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:00.224 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:00.224 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:00.246 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:00.248 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:00.249 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:00.249 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:00.245+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:00.349 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:00.345+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:00.369 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:00.370 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:00.383 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-18T00:59:00.377+0000 7fdb35eeb640 -1 mon.a@0(leader).osd e91 definitely_dead 0 2024-09-18T00:59:00.384 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:00.450 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:00.445+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:00.504 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:00.504 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:00.518 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:00.562 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:00.557+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:00.652 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:00.645+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:00.681 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:00.681 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:00.695 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:00.752 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:00.749+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:00.818 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:00.831 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:00.853 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:00.849+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:00.948 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:00.949 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:00.954 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:00.949+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:00.963 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:00.970 INFO:teuthology.orchestra.run.smithi042.stderr:marked out osd.3. 2024-09-18T00:59:01.055 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:01.049+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:01.087 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:01.087 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:01.103 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:01.155 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:01.149+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:01.228 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:01.229 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:01.242 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:01.290 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:01.285+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:01.356 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:01.349+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:01.380 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:01.380 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:01.395 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:01.482 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:01.477+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:01.558 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:01.553+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:01.563 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:01.563 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:01.578 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:01.666 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:01.661+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:01.750 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:01.751 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:01.760 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:01.753+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:01.765 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:01.860 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:01.853+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:01.939 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:01.939 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:01.953 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:01.998 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:01.993+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:02.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:59:02.085 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:02.085 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:02.094 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:02.089+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:02.099 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:02.162 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:02.157+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:02.223 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:02.223 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:02.237 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:02.264 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:02.257+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:02.272 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:02.364 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:02.357+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:02.400 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:02.400 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:02.414 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:02.420 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:02.420 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:02.434 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:02.452 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:02.453 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:02.470 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:02.471 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:02.489 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:02.489 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:02.501 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:02.497+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:02.507 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:02.507 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:02.525 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:02.544 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:02.544 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:02.565 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:02.557+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:02.584 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:02.584 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:02.584 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:02.584 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:02.598 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:02.685 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:02.681+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:02.765 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:02.765 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:02.765 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:02.765 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:02.765 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:02.767 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:02.761+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:02.779 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:02.868 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:02.865+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:02.915 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:59:02.966 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:02.966 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.555511675999995, 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:02.967 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T00:59:02.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T00:59:02.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:02.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.972 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:26797 3.0 3:03a9420d:::rbd_data.117c4c7db1a0.0000000000000071:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.777084877, 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073789632999999993, 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:02.973 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 26797 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187640+0000", 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187653+0000", 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2357e-05 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187684+0000", 2024-09-18T00:59:02.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1582999999999999e-05 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187699+0000", 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5003e-05 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187806+0000", 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106776 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:02.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.188026+0000", 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021975799999999999 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7", 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.201663+0000", 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013637768 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261226+0000", 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059562916 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261261+0000", 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4360000000000003e-05 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261354+0000", 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3041999999999995e-05 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261427+0000", 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3553000000000005e-05 2024-09-18T00:59:02.978 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64346 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375833290999999, 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048905196999999997, 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:02.979 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64346 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588889+0000", 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588890+0000", 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3e-07 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588893+0000", 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.106e-06 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588904+0000", 2024-09-18T00:59:02.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1037e-05 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588911+0000", 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2729999999999997e-06 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637426+0000", 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048514483999999997 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637537+0000", 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110901 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637795+0000", 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025776599999999998 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64347 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:02.983 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375801418000002, 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049469502999999998, 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64347 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588921+0000", 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588922+0000", 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4300000000000003e-07 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588925+0000", 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588934+0000", 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0329999999999997e-06 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588946+0000", 2024-09-18T00:59:02.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1547e-05 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.637872+0000", 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048926202000000002 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638047+0000", 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174566 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:02.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638391+0000", 2024-09-18T00:59:02.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034401499999999998 2024-09-18T00:59:02.988 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.988 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:02.988 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.988 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64348 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:02.988 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:59:02.988 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375760548000002, 2024-09-18T00:59:02.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050239487999999999, 2024-09-18T00:59:02.989 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:02.989 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:02.989 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:02.989 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:02.989 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:02.989 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:02.989 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64348 2024-09-18T00:59:02.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.989 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:02.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588962+0000", 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588963+0000", 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.133e-06 2024-09-18T00:59:02.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588969+0000", 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4229999999999996e-06 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.588996+0000", 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7396e-05 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:02.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589163+0000", 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016677900000000001 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638498+0000", 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049334743 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.638715+0000", 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000217414 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639202+0000", 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048660000000000001 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64349 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375531643000002, 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050607656000000001, 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:02.993 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64349 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589191+0000", 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589192+0000", 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.536e-06 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589201+0000", 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-06 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:02.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589222+0000", 2024-09-18T00:59:02.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-18T00:59:02.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:02.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589241+0000", 2024-09-18T00:59:02.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8960000000000001e-05 2024-09-18T00:59:02.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:02.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639285+0000", 2024-09-18T00:59:02.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050043892999999999 2024-09-18T00:59:02.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:02.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639457+0000", 2024-09-18T00:59:02.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017286299999999999 2024-09-18T00:59:02.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639799+0000", 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034117400000000002 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64350 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375461297999998, 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051028502000000003, 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:02.998 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64350 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:02.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589261+0000", 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589262+0000", 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1570000000000001e-06 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589280+0000", 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7943000000000001e-05 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589299+0000", 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8940000000000002e-05 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589316+0000", 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6654e-05 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639872+0000", 2024-09-18T00:59:03.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050556151000000001 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.639988+0000", 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115423 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640290+0000", 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030223400000000002 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64351 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1048576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375385557999998, 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051389975999999997, 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64351 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.003 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:59:03.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589337+0000", 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589338+0000", 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589345+0000", 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589363+0000", 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8346e-05 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589381+0000", 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7357e-05 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640373+0000", 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.050992347 2024-09-18T00:59:03.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640504+0000", 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000131011 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640727+0000", 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022310799999999999 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64352 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375322029000003, 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051932649999999997, 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64352 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.008 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589401+0000", 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589402+0000", 2024-09-18T00:59:03.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.237e-06 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589408+0000", 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2770000000000002e-06 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589426+0000", 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8006e-05 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589441+0000", 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4934e-05 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640803+0000", 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051361619999999997 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.640986+0000", 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018303199999999999 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641333+0000", 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034754399999999998 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64353 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375263635000003, 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052565148999999999, 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.012 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64353 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589459+0000", 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589460+0000", 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4300000000000001e-06 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589466+0000", 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.446e-06 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589485+0000", 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8726999999999999e-05 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589499+0000", 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.402e-05 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641427+0000", 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051927945000000003 2024-09-18T00:59:03.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.641601+0000", 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174859 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642024+0000", 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000422722 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64354 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375188936000001, 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053055607999999997, 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64354 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589534+0000", 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589535+0000", 2024-09-18T00:59:03.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3430000000000001e-06 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589542+0000", 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589561+0000", 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9207000000000001e-05 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589577+0000", 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5933000000000001e-05 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642111+0000", 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052533799999999999 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642271+0000", 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015998499999999999 2024-09-18T00:59:03.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642589+0000", 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031818999999999999 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64355 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375124900000003, 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053570327000000001, 2024-09-18T00:59:03.021 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64355 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589598+0000", 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589599+0000", 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589605+0000", 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8830000000000003e-06 2024-09-18T00:59:03.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589622+0000", 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.732e-05 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589636+0000", 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3703000000000001e-05 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642672+0000", 2024-09-18T00:59:03.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053036204000000003 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.642828+0000", 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155451 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643168+0000", 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034051900000000001 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64356 3.1e 3:7f0d8b97:::rbd_data.117c689d3abc.0000000000000002:17 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375067387000001, 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054203299000000003, 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64356 2024-09-18T00:59:03.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589655+0000", 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589656+0000", 2024-09-18T00:59:03.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589662+0000", 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9800000000000003e-06 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589681+0000", 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8300000000000001e-05 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589695+0000", 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3997e-05 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643249+0000", 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053554101999999999 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643423+0000", 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174062 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643859+0000", 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043562499999999998 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64357 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.375009437000003, 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054239849, 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:03.031 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.031 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.031 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.031 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64357 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589713+0000", 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589714+0000", 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.266e-06 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589720+0000", 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2340000000000002e-06 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589738+0000", 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7932999999999999e-05 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589752+0000", 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4867e-05 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643934+0000", 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054181721000000002 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.643953+0000", 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8828e-05 2024-09-18T00:59:03.035 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64358 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.374951557999999, 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054261125, 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.036 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64358 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589772+0000", 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1170000000000001e-06 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589778+0000", 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8429999999999998e-06 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589799+0000", 2024-09-18T00:59:03.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0727e-05 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589813+0000", 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4403000000000001e-05 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644021+0000", 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054207784000000002 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644032+0000", 2024-09-18T00:59:03.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1251e-05 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64359 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.374889592000002, 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286586999999997, 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:03.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64359 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:59:03.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589834+0000", 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589841+0000", 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.956e-06 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589864+0000", 2024-09-18T00:59:03.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3124000000000001e-05 2024-09-18T00:59:03.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589879+0000", 2024-09-18T00:59:03.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.486e-05 2024-09-18T00:59:03.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644099+0000", 2024-09-18T00:59:03.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054219429 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644120+0000", 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1004000000000001e-05 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64360 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.044 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.374825397999999, 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054290699999999997, 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64360 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589898+0000", 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3600000000000002e-07 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589903+0000", 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4669999999999998e-06 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589913+0000", 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0463e-05 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589921+0000", 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7959999999999996e-06 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644173+0000", 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054251903999999997 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644188+0000", 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5133999999999999e-05 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.048 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64361 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.374791919000003, 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054320101000000003, 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64361 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5000000000000002e-07 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589935+0000", 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2799999999999999e-06 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589944+0000", 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589954+0000", 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0036e-05 2024-09-18T00:59:03.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644240+0000", 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286273000000003 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644251+0000", 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0392e-05 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64362 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.374758773000003, 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054340196, 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64362 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:59:03.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589978+0000", 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3570000000000001e-05 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589991+0000", 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3196999999999999e-05 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590000+0000", 2024-09-18T00:59:03.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2800000000000003e-06 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644294+0000", 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054294873 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644304+0000", 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6560000000000004e-06 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64363 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.374712743000003, 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054356660000000001, 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64363 2024-09-18T00:59:03.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590011+0000", 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7400000000000003e-07 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590014+0000", 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590024+0000", 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5670000000000001e-06 2024-09-18T00:59:03.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590031+0000", 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3830000000000003e-06 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644354+0000", 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054323429999999999 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644367+0000", 2024-09-18T00:59:03.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.217e-05 2024-09-18T00:59:03.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.061 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.061 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.061 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:03.062 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:03.063 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:03.057+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:03.085 INFO:teuthology.orchestra.run.smithi042.stdout:3707 writesame 0xa97322d thru 0xa9759eb data_size 0x19(0x27bf bytes) 2024-09-18T00:59:03.099 INFO:teuthology.orchestra.run.smithi042.stdout:3708 read 0x842f34c thru 0x84393d1 (0xa086 bytes) 2024-09-18T00:59:03.102 INFO:teuthology.orchestra.run.smithi042.stdout:3710 write 0x4c10490 thru 0x4c1683a (0x63ab bytes) 2024-09-18T00:59:03.123 INFO:teuthology.orchestra.run.smithi042.stdout:3711 trunc from 0xcc670a3 to 0xa04f64a 2024-09-18T00:59:03.158 INFO:teuthology.orchestra.run.smithi042.stdout:3714 trunc from 0xa04f64a to 0x1c15209 2024-09-18T00:59:03.163 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:03.157+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:03.266 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:03.257+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:03.289 INFO:teuthology.orchestra.run.smithi042.stdout:3715 writesame 0x1e7d7de thru 0x1e8821b data_size 0x4d(0xaa3e bytes) 2024-09-18T00:59:03.289 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:03.289 INFO:teuthology.orchestra.run.smithi042.stdout:3717 punch from 0x11a48d4 to 0x11ab76f, (0x6e9b bytes) 2024-09-18T00:59:03.289 INFO:teuthology.orchestra.run.smithi042.stdout:3722 clone 31 order 24 su 524288 sc 5 2024-09-18T00:59:03.302 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:03.302 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:03.302 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:03.302 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:03.320 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:03.365 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:03.361+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:03.465 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:03.461+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:03.503 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:03.503 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:03.503 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:03.503 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:03.503 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:03.516 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:03.605 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:03.601+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:03.668 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:03.665+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:03.677 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:03.677 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:03.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:03.677 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:03.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.677 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T00:59:03.677 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:03.678 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.585955810000002, 2024-09-18T00:59:03.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T00:59:03.678 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.678 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.678 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:03.678 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:03.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T00:59:03.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T00:59:03.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T00:59:03.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T00:59:03.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T00:59:03.682 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.195060858999994, 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T00:59:03.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T00:59:03.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:03.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T00:59:03.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T00:59:03.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T00:59:03.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T00:59:03.793 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.793 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.793 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.793 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78518 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1183744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:03.793 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:03.793 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.84353003, 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054532499999999998, 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78518 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831040+0000", 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3099999999999997e-07 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831042+0000", 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2879999999999999e-06 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831050+0000", 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.464e-06 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831055+0000", 2024-09-18T00:59:03.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0819999999999998e-06 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885354+0000", 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298739999999998 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885428+0000", 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4567999999999995e-05 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885572+0000", 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014372699999999999 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78519 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1204224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.843508181000001, 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054707261, 2024-09-18T00:59:03.798 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78519 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831062+0000", 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831064+0000", 2024-09-18T00:59:03.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0360000000000001e-06 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831069+0000", 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4149999999999998e-06 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831075+0000", 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2249999999999997e-06 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885602+0000", 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054526893 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885673+0000", 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0865000000000007e-05 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885768+0000", 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5422999999999998e-05 2024-09-18T00:59:03.802 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78520 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1224704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.843487724999999, 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054906695999999998, 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.803 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78520 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:03.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831084+0000", 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2960000000000001e-06 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831091+0000", 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831095+0000", 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.792e-06 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885802+0000", 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054706795000000003 2024-09-18T00:59:03.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885884+0000", 2024-09-18T00:59:03.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2297000000000002e-05 2024-09-18T00:59:03.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885988+0000", 2024-09-18T00:59:03.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010393199999999999 2024-09-18T00:59:03.807 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.807 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78521 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.843442851999999, 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055066616999999998, 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.808 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78521 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:03.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831127+0000", 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.99e-07 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831130+0000", 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6819999999999999e-06 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831137+0000", 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3849999999999996e-06 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831145+0000", 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7360000000000001e-06 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886020+0000", 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054874700999999998 2024-09-18T00:59:03.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886096+0000", 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6234000000000002e-05 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886193+0000", 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7280000000000004e-05 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78522 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.843417090999999, 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055137938999999997, 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78522 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.813 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831153+0000", 2024-09-18T00:59:03.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.37e-07 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831155+0000", 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4420000000000001e-06 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831161+0000", 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831166+0000", 2024-09-18T00:59:03.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8749999999999999e-06 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886279+0000", 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055112337999999997 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886290+0000", 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1527e-05 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.816 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78523 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.843396411000001, 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055171425000000003, 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.817 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78523 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0900000000000002e-07 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831176+0000", 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4830000000000002e-06 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831183+0000", 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0679999999999999e-06 2024-09-18T00:59:03.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831189+0000", 2024-09-18T00:59:03.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:03.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886331+0000", 2024-09-18T00:59:03.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055142669999999998 2024-09-18T00:59:03.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886344+0000", 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2918999999999999e-05 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78524 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:03.821 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.843374295, 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055194430000000003, 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78524 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831196+0000", 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1200000000000003e-07 2024-09-18T00:59:03.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831198+0000", 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831204+0000", 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2740000000000004e-06 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831209+0000", 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0900000000000004e-06 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886381+0000", 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055172142 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886389+0000", 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2150000000000001e-06 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.825 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78525 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.843353782000001, 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055206535000000001, 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.826 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78525 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832358+0000", 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001142531 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832364+0000", 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832375+0000", 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1381e-05 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832382+0000", 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9800000000000001e-06 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886415+0000", 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032678000000001 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886422+0000", 2024-09-18T00:59:03.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4449999999999999e-06 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108130 3.1 3:83363b4a:::rbd_data.117c8e0d79e4.000000000000002d:20 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.48063871200000002, 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054351575999999999, 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.830 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108130 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:03.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1100000000000001e-07 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193933+0000", 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3690000000000001e-06 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193940+0000", 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1460000000000001e-06 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193944+0000", 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9230000000000001e-06 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.245560+0000", 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051615019999999998 2024-09-18T00:59:03.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.245633+0000", 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3490000000000003e-05 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.248282+0000", 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026492170000000002 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108393 3.1 3:86b5547c:::rbd_data.117c8e0d79e4.0000000000000037:20 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.477547157, 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054339392, 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108393 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.835 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5499999999999999e-07 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197024+0000", 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.308e-06 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197029+0000", 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2024-09-18T00:59:03.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197032+0000", 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.994e-06 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251285+0000", 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054253006999999999 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251322+0000", 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7104000000000003e-05 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251362+0000", 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9941000000000003e-05 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108394 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:03.839 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.47748475099999999, 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054381878000000002, 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108394 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:03.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5400000000000001e-07 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197088+0000", 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8830000000000002e-06 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197096+0000", 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8560000000000007e-06 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197101+0000", 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251388+0000", 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054287588999999997 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251427+0000", 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8665999999999999e-05 2024-09-18T00:59:03.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251466+0000", 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9369000000000002e-05 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108395 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:03.844 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.47745933600000001, 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054457976999999998, 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108395 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:03.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197111+0000", 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6000000000000004e-07 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197113+0000", 2024-09-18T00:59:03.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4119999999999999e-06 2024-09-18T00:59:03.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197119+0000", 2024-09-18T00:59:03.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4160000000000001e-06 2024-09-18T00:59:03.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197125+0000", 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8320000000000002e-06 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251492+0000", 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054366997 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251529+0000", 2024-09-18T00:59:03.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6563000000000002e-05 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251568+0000", 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9196999999999998e-05 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108396 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:03.849 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.47743626500000003, 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054537624, 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108396 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:03.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6199999999999999e-07 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197135+0000", 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197139+0000", 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1749999999999998e-06 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197143+0000", 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1350000000000001e-06 2024-09-18T00:59:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251594+0000", 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054451160999999998 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251632+0000", 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7840999999999999e-05 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251671+0000", 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9169999999999999e-05 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108413 3.1 3:8453e3d7:::rbd_data.117c8e0d79e4.0000000000000022:20 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.47699225699999998, 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054401959, 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.854 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108413 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197578+0000", 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7199999999999999e-07 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197581+0000", 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8439999999999998e-06 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197587+0000", 2024-09-18T00:59:03.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1759999999999998e-06 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197592+0000", 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4310000000000002e-06 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251903+0000", 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054310418999999999 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251939+0000", 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6310999999999998e-05 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251979+0000", 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0206000000000001e-05 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108428 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:03.858 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.47679781900000001, 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054482228000000001, 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108428 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9900000000000001e-07 2024-09-18T00:59:03.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197774+0000", 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.373e-06 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197781+0000", 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3130000000000004e-06 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197786+0000", 2024-09-18T00:59:03.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9980000000000003e-06 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252117+0000", 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054331329999999997 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252178+0000", 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0952000000000002e-05 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252254+0000", 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5863000000000002e-05 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108429 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.47677687000000002, 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054624675999999997, 2024-09-18T00:59:03.863 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108429 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197793+0000", 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0600000000000001e-07 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197795+0000", 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.458e-06 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197802+0000", 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.691e-06 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197807+0000", 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2249999999999999e-06 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252283+0000", 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054476138 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252348+0000", 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4783999999999995e-05 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252417+0000", 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8973999999999994e-05 2024-09-18T00:59:03.867 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108430 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.47675467900000001, 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054759302000000003, 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108430 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:03.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.58e-07 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197818+0000", 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5600000000000001e-06 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197824+0000", 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.511e-06 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197829+0000", 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1619999999999999e-06 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252446+0000", 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054616988999999998 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252513+0000", 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6558999999999998e-05 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252574+0000", 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1062999999999995e-05 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.872 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108540 3.1 3:83363b4a:::rbd_data.117c8e0d79e4.000000000000002d:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.4693753, 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056511851000000002, 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108540 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.873 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.01e-07 2024-09-18T00:59:03.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205197+0000", 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205204+0000", 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4610000000000002e-06 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205210+0000", 2024-09-18T00:59:03.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5140000000000001e-06 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.256876+0000", 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051666493000000001 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.256915+0000", 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8724999999999999e-05 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:03.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:03.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.261706+0000", 2024-09-18T00:59:03.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004791086 2024-09-18T00:59:03.877 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.877 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.877 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.877 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:03.877 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:03.877 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:03.879 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:03.879 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:03.877+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:03.980 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:03.973+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:04.057 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:04.057 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:04.071 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:04.081 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:04.077+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:04.182 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:04.177+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:04.230 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:04.230 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:04.230 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:04.230 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:04.231 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:04.244 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:04.289 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:04.285+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:04.383 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:04.377+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:04.415 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:04.415 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:04.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:04.415 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:04.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.415 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4041 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:04.415 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:04.415 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.410481395999994, 2024-09-18T00:59:04.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037088000000000003, 2024-09-18T00:59:04.415 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4041 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003005+0000", 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.187e-06 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003011+0000", 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003029+0000", 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8026999999999999e-05 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003043+0000", 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039484+0000", 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036440895000000001 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039722+0000", 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000238507 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040092+0000", 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000369818 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.419 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.410423773000005, 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.420 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:59:04.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:04.424 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.410365515999999, 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:04.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.428 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.410307076999999, 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.429 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:59:04.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:04.433 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.410245860000003, 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:04.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.437 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.410155445000001, 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:59:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.410084511999997, 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:59:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:59:04.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:59:04.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.410024906000004, 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.447 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:04.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:04.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:59:04.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.409965536000001, 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.452 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:59:04.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:59:04.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.409903747000001, 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:59:04.456 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:59:04.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:59:04.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.409770924, 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.461 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:59:04.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:04.465 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 75.854553447000001, 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.466 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T00:59:04.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T00:59:04.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.471 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 75.854053323000002, 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.472 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T00:59:04.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 75.854022814999993, 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T00:59:04.476 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.480 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 75.853998149999995, 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.481 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T00:59:04.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 75.853969617999994, 2024-09-18T00:59:04.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T00:59:04.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.489 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 75.853636327000004, 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.490 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T00:59:04.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 75.853594475999998, 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T00:59:04.494 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T00:59:04.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 75.852556980000003, 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T00:59:04.500 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T00:59:04.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.504 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.200268975, 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:04.505 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:04.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T00:59:04.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:04.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.510 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.511 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.511 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:04.511 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:04.511 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:04.512 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:04.505+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:04.512 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:04.562 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:04.613 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:04.613+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:04.651 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:04.651 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:04.651 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:04.651 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:04.666 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:04.711 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:04.711 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:04.713 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:04.709+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:04.728 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:04.729 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:04.746 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:04.747 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:04.766 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:04.766 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:04.786 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:04.786 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:04.792 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:04.792 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:04.806 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:04.808 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:04.808 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:04.829 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:04.829 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:04.829 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:04.821+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:04.867 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:04.867 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:04.934 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:04.929+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:04.944 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:04.944 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:04.959 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:04.961 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:59:05.046 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:05.041+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:05.062 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone30 from 0x1e8821c (overlap 0x1c15209) to 0x194636e 2024-09-18T00:59:05.117 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:05.113+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:05.145 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:05.146 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:05.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:05.228 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-18T00:59:05.228 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:05.221+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:05.319 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:05.313+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:05.378 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:05.378 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:05.399 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:05.446 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:05.441+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:05.521 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:05.517+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:05.563 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:05.563 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:05.578 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:05.622 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:05.617+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:05.723 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:05.717+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:05.766 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:05.766 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:05.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:05.826 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:05.821+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:05.893 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:05.893 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:05.906 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:05.935 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:05.921+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:05.995 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:59:05.999 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7] out_osds: [2, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:59:05.999 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:59:05.999 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T00:59:05.999 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T00:59:06.025 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:06.021+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:06.084 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:06.084 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:06.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:06.146 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:06.141+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:06.210 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:06.210 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:06.222 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:06.266 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:06.261+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 97, 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "82a96d76-ba83-49e2-a985-84604bd599ba", 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-18T00:57:07.138732+0000", 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-18T00:59:05.985389+0000", 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-18T00:57:20.258605+0000", 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-18T00:59:00.963633+0000", 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5804032, 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-18T00:59:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "jewel", 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-18T00:59:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:21.962521+0000", 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "19", 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 8, 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 8, 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.25, 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-18T00:59:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:25.721733+0000", 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:06.301 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "23", 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 23, 2024-09-18T00:59:06.302 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:06.303 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 3, 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 3, 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.75, 2024-09-18T00:59:06.305 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2.25, 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2.25, 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:35.599413+0000", 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:06.306 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-18T00:59:06.307 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "95", 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:06.308 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 33, 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 95, 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:06.309 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:06.310 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 2, 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 2, 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.75, 2024-09-18T00:59:06.311 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.5, 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.5, 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "3f359ce5-6f6e-4161-8554-8ac028347274", 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:06.312 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 95, 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.316 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/199763443", 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/199763443", 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/199763443", 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/199763443", 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:06.317 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "48687611-1556-4b6d-86af-7872494a91d7", 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:06.318 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 95, 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.319 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/345149800", 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/345149800", 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/345149800", 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/345149800", 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "9e6b1f90-ce9b-46d4-92fe-d457f7e415d5", 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:06.323 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.327 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/431239931", 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/431239931", 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/431239931", 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/431239931", 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5", 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 89, 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.330 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.331 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.332 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/1775474180", 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/1775474180", 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/1775474180", 2024-09-18T00:59:06.333 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/1775474180", 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "802ccbeb-7351-4b38-832a-695117c7e493", 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:06.334 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 95, 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6808", 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6809", 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6810", 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.336 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6811", 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6814", 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.337 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6815", 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6812", 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6813", 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:06.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6809/4198401207", 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6811/4198401207", 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6815/4198401207", 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6813/4198401207", 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:59:06.339 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d958b3da-efa1-4d86-b81d-e78874441437", 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 95, 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.340 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6824", 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6825", 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.341 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6826", 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6827", 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6830", 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6831", 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6828", 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6829", 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6825/2599825823", 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6827/2599825823", 2024-09-18T00:59:06.344 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6831/2599825823", 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6829/2599825823", 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "51b3cea8-c686-4fd3-b3f4-ba936cf26b64", 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 95, 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6800", 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6801", 2024-09-18T00:59:06.346 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6802", 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6803", 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:06.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6806", 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6807", 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.348 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6804", 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6805", 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:06.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6801/3550656451", 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6803/3550656451", 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6807/3550656451", 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6805/3550656451", 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.350 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b02ead1c-837a-4b07-9ad1-4e903ed5bc4e", 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 24, 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6816", 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6817", 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6818", 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6819", 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6822", 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6823", 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:06.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6820", 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6821", 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6817/2993352359", 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6819/2993352359", 2024-09-18T00:59:06.355 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6823/2993352359", 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6821/2993352359", 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:06.356 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.896477+0000", 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.941756+0000", 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:06.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.560180+0000", 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:59:06.358 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.077394+0000", 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.996264+0000", 2024-09-18T00:59:06.359 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.511686+0000", 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:59:06.360 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.411051+0000", 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:06.361 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.888092+0000", 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-18T00:59:06.362 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [], 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-18T00:59:06.363 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-18T00:59:06.364 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-18T00:59:06.364 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-18T00:59:06.364 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-18T00:59:06.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:06.364 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:06.366 INFO:tasks.thrashosds.thrasher:j is {'epoch': 97, 'fsid': '82a96d76-ba83-49e2-a985-84604bd599ba', 'created': '2024-09-18T00:57:07.138732+0000', 'modified': '2024-09-18T00:59:05.985389+0000', 'last_up_change': '2024-09-18T00:57:20.258605+0000', 'last_in_change': '2024-09-18T00:59:00.963633+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T00:57:21.962521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T00:57:25.721733+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T00:57:35.599413+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3f359ce5-6f6e-4161-8554-8ac028347274', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 199763443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 199763443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 199763443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 199763443}]}, 'public_addr': '172.21.15.42:6801/199763443', 'cluster_addr': '172.21.15.42:6803/199763443', 'heartbeat_back_addr': '172.21.15.42:6807/199763443', 'heartbeat_front_addr': '172.21.15.42:6805/199763443', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '48687611-1556-4b6d-86af-7872494a91d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 345149800}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 345149800}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 345149800}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 345149800}]}, 'public_addr': '172.21.15.42:6809/345149800', 'cluster_addr': '172.21.15.42:6811/345149800', 'heartbeat_back_addr': '172.21.15.42:6815/345149800', 'heartbeat_front_addr': '172.21.15.42:6813/345149800', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9e6b1f90-ce9b-46d4-92fe-d457f7e415d5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 431239931}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 431239931}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 431239931}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 431239931}]}, 'public_addr': '172.21.15.42:6825/431239931', 'cluster_addr': '172.21.15.42:6827/431239931', 'heartbeat_back_addr': '172.21.15.42:6831/431239931', 'heartbeat_front_addr': '172.21.15.42:6829/431239931', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 89, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 1775474180}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 1775474180}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 1775474180}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 1775474180}]}, 'public_addr': '172.21.15.42:6817/1775474180', 'cluster_addr': '172.21.15.42:6819/1775474180', 'heartbeat_back_addr': '172.21.15.42:6823/1775474180', 'heartbeat_front_addr': '172.21.15.42:6821/1775474180', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '802ccbeb-7351-4b38-832a-695117c7e493', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6809', 'nonce': 4198401207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6811', 'nonce': 4198401207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6815', 'nonce': 4198401207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6813', 'nonce': 4198401207}]}, 'public_addr': '172.21.15.195:6809/4198401207', 'cluster_addr': '172.21.15.195:6811/4198401207', 'heartbeat_back_addr': '172.21.15.195:6815/4198401207', 'heartbeat_front_addr': '172.21.15.195:6813/4198401207', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd958b3da-efa1-4d86-b81d-e78874441437', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6824', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6825', 'nonce': 2599825823}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6826', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6827', 'nonce': 2599825823}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6830', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6831', 'nonce': 2599825823}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6828', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6829', 'nonce': 2599825823}]}, 'public_addr': '172.21.15.195:6825/2599825823', 'cluster_addr': '172.21.15.195:6827/2599825823', 'heartbeat_back_addr': '172.21.15.195:6831/2599825823', 'heartbeat_front_addr': '172.21.15.195:6829/2599825823', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '51b3cea8-c686-4fd3-b3f4-ba936cf26b64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6800', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6801', 'nonce': 3550656451}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6803', 'nonce': 3550656451}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6807', 'nonce': 3550656451}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6805', 'nonce': 3550656451}]}, 'public_addr': '172.21.15.195:6801/3550656451', 'cluster_addr': '172.21.15.195:6803/3550656451', 'heartbeat_back_addr': '172.21.15.195:6807/3550656451', 'heartbeat_front_addr': '172.21.15.195:6805/3550656451', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b02ead1c-837a-4b07-9ad1-4e903ed5bc4e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6817', 'nonce': 2993352359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6818', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6819', 'nonce': 2993352359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6822', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6823', 'nonce': 2993352359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6820', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6821', 'nonce': 2993352359}]}, 'public_addr': '172.21.15.195:6817/2993352359', 'cluster_addr': '172.21.15.195:6819/2993352359', 'heartbeat_back_addr': '172.21.15.195:6823/2993352359', 'heartbeat_front_addr': '172.21.15.195:6821/2993352359', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.896477+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.941756+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.560180+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.077394+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.996264+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.511686+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.411051+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.888092+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T00:59:06.366 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:59:06.366 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:06.357+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:06.388 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:06.388 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:06.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:06.490 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:06.485+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:06.567 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:06.561+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:06.574 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:06.574 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:06.589 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:06.669 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:06.661+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:06.693 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:59:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:59:06.705 INFO:teuthology.orchestra.run.smithi042.stdout:3726 write 0x659395 thru 0x666875 (0xd4e1 bytes) 2024-09-18T00:59:06.713 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-18T00:59:05.994066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":940049768,"num_objects":1880,"num_object_clones":420,"num_object_copies":3760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1880,"num_whiteouts":837,"num_read":145183,"num_read_kb":684842,"num_write":14324,"num_write_kb":478077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2926,"num_bytes_recovered":207217829,"num_keys_recovered":402,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11680,"ondisk_log_size":11680,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":690820,"kb_used_data":485636,"kb_used_omap":9,"kb_used_meta":205046,"kb_avail":565540220,"statfs":{"total":579820584960,"available":579113185280,"internally_reserved":0,"allocated":497291264,"data_stored":599055484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9564,"internal_metadata":209967780},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13718093,"num_objects":44,"num_object_clones":5,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":12123,"num_read_kb":27162,"num_write":257,"num_write_kb":9313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":774,"num_bytes_recovered":53269132,"num_keys_recovered":153,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.086998"},"pg_stats":[{"pgid":"3.1f","version":"85'250","reported_seq":1590,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:04.987346+0000","last_change":"2024-09-18T00:57:38.909849+0000","last_active":"2024-09-18T00:59:04.987346+0000","last_peered":"2024-09-18T00:59:04.987346+0000","last_clean":"2024-09-18T00:59:04.987346+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T00:59:04.987346+0000","last_undegraded":"2024-09-18T00:59:04.987346+0000","last_fullsized":"2024-09-18T00:59:04.987346+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:48:03.566042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001565786,"stat_sum":{"num_bytes":12596921,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":27,"num_read":2526,"num_read_kb":6489,"num_write":214,"num_write_kb":5416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"96'358","reported_seq":3525,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.077448+0000","last_change":"2024-09-18T00:59:04.996334+0000","last_active":"2024-09-18T00:59:05.077448+0000","last_peered":"2024-09-18T00:59:05.077448+0000","last_clean":"2024-09-18T00:59:05.077448+0000","last_became_active":"2024-09-18T00:59:04.993158+0000","last_became_peered":"2024-09-18T00:59:04.993158+0000","last_unstale":"2024-09-18T00:59:05.077448+0000","last_undegraded":"2024-09-18T00:59:05.077448+0000","last_fullsized":"2024-09-18T00:59:05.077448+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:54.846283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040574399999999998,"stat_sum":{"num_bytes":38682782,"num_objects":76,"num_object_clones":17,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":36,"num_read":3214,"num_read_kb":19838,"num_write":416,"num_write_kb":9899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":26102183,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"96'614","reported_seq":11033,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.598504+0000","last_change":"2024-09-18T00:59:04.984783+0000","last_active":"2024-09-18T00:59:05.598504+0000","last_peered":"2024-09-18T00:59:05.598504+0000","last_clean":"2024-09-18T00:59:05.598504+0000","last_became_active":"2024-09-18T00:59:04.983725+0000","last_became_peered":"2024-09-18T00:59:04.983725+0000","last_unstale":"2024-09-18T00:59:05.598504+0000","last_undegraded":"2024-09-18T00:59:05.598504+0000","last_fullsized":"2024-09-18T00:59:05.598504+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:56:14.105137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039089,"stat_sum":{"num_bytes":40034659,"num_objects":69,"num_object_clones":18,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":34,"num_read":12768,"num_read_kb":37358,"num_write":824,"num_write_kb":7018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":4309906,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"96'432","reported_seq":2731,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.638048+0000","last_change":"2024-09-18T00:57:38.901600+0000","last_active":"2024-09-18T00:59:05.638048+0000","last_peered":"2024-09-18T00:59:05.638048+0000","last_clean":"2024-09-18T00:59:05.638048+0000","last_became_active":"2024-09-18T00:57:36.906003+0000","last_became_peered":"2024-09-18T00:57:36.906003+0000","last_unstale":"2024-09-18T00:59:05.638048+0000","last_undegraded":"2024-09-18T00:59:05.638048+0000","last_fullsized":"2024-09-18T00:59:05.638048+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:30.493327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000711266,"stat_sum":{"num_bytes":18082453,"num_objects":66,"num_object_clones":15,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":29,"num_read":3614,"num_read_kb":21716,"num_write":498,"num_write_kb":6426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"92'574","reported_seq":7636,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.496351+0000","last_change":"2024-09-18T00:57:38.921854+0000","last_active":"2024-09-18T00:59:05.496351+0000","last_peered":"2024-09-18T00:59:05.496351+0000","last_clean":"2024-09-18T00:59:05.496351+0000","last_became_active":"2024-09-18T00:57:36.913393+0000","last_became_peered":"2024-09-18T00:57:36.913393+0000","last_unstale":"2024-09-18T00:59:05.496351+0000","last_undegraded":"2024-09-18T00:59:05.496351+0000","last_fullsized":"2024-09-18T00:59:05.496351+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:21.406298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002554022,"stat_sum":{"num_bytes":42660436,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":24,"num_read":9833,"num_read_kb":27919,"num_write":709,"num_write_kb":14472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"92'241","reported_seq":3307,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.734895+0000","last_change":"2024-09-18T00:57:38.901805+0000","last_active":"2024-09-18T00:59:05.734895+0000","last_peered":"2024-09-18T00:59:05.734895+0000","last_clean":"2024-09-18T00:59:05.734895+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T00:59:05.734895+0000","last_undegraded":"2024-09-18T00:59:05.734895+0000","last_fullsized":"2024-09-18T00:59:05.734895+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:19.432602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098070899999999996,"stat_sum":{"num_bytes":38696822,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":3880,"num_read_kb":26425,"num_write":235,"num_write_kb":8993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"94'337","reported_seq":27046,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.762962+0000","last_change":"2024-09-18T00:57:38.922010+0000","last_active":"2024-09-18T00:59:05.762962+0000","last_peered":"2024-09-18T00:59:05.762962+0000","last_clean":"2024-09-18T00:59:05.762962+0000","last_became_active":"2024-09-18T00:57:36.910457+0000","last_became_peered":"2024-09-18T00:57:36.910457+0000","last_unstale":"2024-09-18T00:59:05.762962+0000","last_undegraded":"2024-09-18T00:59:05.762962+0000","last_fullsized":"2024-09-18T00:59:05.762962+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:26:58.517443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026842849999999998,"stat_sum":{"num_bytes":79557268,"num_objects":68,"num_object_clones":15,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":29,"num_read":26658,"num_read_kb":71735,"num_write":337,"num_write_kb":22052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"88'507","reported_seq":2154,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.591458+0000","last_change":"2024-09-18T00:57:38.908144+0000","last_active":"2024-09-18T00:59:05.591458+0000","last_peered":"2024-09-18T00:59:05.591458+0000","last_clean":"2024-09-18T00:59:05.591458+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T00:59:05.591458+0000","last_undegraded":"2024-09-18T00:59:05.591458+0000","last_fullsized":"2024-09-18T00:59:05.591458+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:56.555331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000871597,"stat_sum":{"num_bytes":41430770,"num_objects":58,"num_object_clones":11,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":3338,"num_read_kb":7188,"num_write":676,"num_write_kb":35802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"96'483","reported_seq":5020,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.761105+0000","last_change":"2024-09-18T00:59:04.992114+0000","last_active":"2024-09-18T00:59:05.761105+0000","last_peered":"2024-09-18T00:59:05.761105+0000","last_clean":"2024-09-18T00:59:05.761105+0000","last_became_active":"2024-09-18T00:59:04.989077+0000","last_became_peered":"2024-09-18T00:59:04.989077+0000","last_unstale":"2024-09-18T00:59:05.761105+0000","last_undegraded":"2024-09-18T00:59:05.761105+0000","last_fullsized":"2024-09-18T00:59:05.761105+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:14:59.826664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001269769,"stat_sum":{"num_bytes":36705092,"num_objects":59,"num_object_clones":13,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":20,"num_read":5611,"num_read_kb":31496,"num_write":588,"num_write_kb":17533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":7794778,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"94'311","reported_seq":4177,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.759614+0000","last_change":"2024-09-18T00:59:04.994621+0000","last_active":"2024-09-18T00:59:05.759614+0000","last_peered":"2024-09-18T00:59:05.759614+0000","last_clean":"2024-09-18T00:59:05.759614+0000","last_became_active":"2024-09-18T00:59:04.991534+0000","last_became_peered":"2024-09-18T00:59:04.991534+0000","last_unstale":"2024-09-18T00:59:05.759614+0000","last_undegraded":"2024-09-18T00:59:05.759614+0000","last_fullsized":"2024-09-18T00:59:05.759614+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:27:28.030758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076552,"stat_sum":{"num_bytes":16006336,"num_objects":63,"num_object_clones":18,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":24,"num_read":3937,"num_read_kb":18254,"num_write":453,"num_write_kb":20030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":16561805,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"96'373","reported_seq":1348,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.589299+0000","last_change":"2024-09-18T00:57:38.909647+0000","last_active":"2024-09-18T00:59:05.589299+0000","last_peered":"2024-09-18T00:59:05.589299+0000","last_clean":"2024-09-18T00:59:05.589299+0000","last_became_active":"2024-09-18T00:57:36.902977+0000","last_became_peered":"2024-09-18T00:57:36.902977+0000","last_unstale":"2024-09-18T00:59:05.589299+0000","last_undegraded":"2024-09-18T00:59:05.589299+0000","last_fullsized":"2024-09-18T00:59:05.589299+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:50.913065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051351900000000004,"stat_sum":{"num_bytes":36925783,"num_objects":57,"num_object_clones":10,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":1541,"num_read_kb":10787,"num_write":560,"num_write_kb":43465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"94'187","reported_seq":1472,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.350725+0000","last_change":"2024-09-18T00:59:04.993102+0000","last_active":"2024-09-18T00:59:05.350725+0000","last_peered":"2024-09-18T00:59:05.350725+0000","last_clean":"2024-09-18T00:59:05.350725+0000","last_became_active":"2024-09-18T00:59:04.992342+0000","last_became_peered":"2024-09-18T00:59:04.992342+0000","last_unstale":"2024-09-18T00:59:05.350725+0000","last_undegraded":"2024-09-18T00:59:05.350725+0000","last_fullsized":"2024-09-18T00:59:05.350725+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:14:12.473451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028223319999999999,"stat_sum":{"num_bytes":3837726,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":23,"num_read":889,"num_read_kb":1195,"num_write":154,"num_write_kb":1150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":2312968,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:04.979525+0000","last_change":"2024-09-18T00:59:01.916258+0000","last_active":"2024-09-18T00:59:04.979525+0000","last_peered":"2024-09-18T00:59:04.979525+0000","last_clean":"2024-09-18T00:59:04.979525+0000","last_became_active":"2024-09-18T00:59:01.915747+0000","last_became_peered":"2024-09-18T00:59:01.915747+0000","last_unstale":"2024-09-18T00:59:04.979525+0000","last_undegraded":"2024-09-18T00:59:04.979525+0000","last_fullsized":"2024-09-18T00:59:04.979525+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:27.934590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"92'496","reported_seq":2396,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.763118+0000","last_change":"2024-09-18T00:57:38.920098+0000","last_active":"2024-09-18T00:59:05.763118+0000","last_peered":"2024-09-18T00:59:05.763118+0000","last_clean":"2024-09-18T00:59:05.763118+0000","last_became_active":"2024-09-18T00:57:36.920599+0000","last_became_peered":"2024-09-18T00:57:36.920599+0000","last_unstale":"2024-09-18T00:59:05.763118+0000","last_undegraded":"2024-09-18T00:59:05.763118+0000","last_fullsized":"2024-09-18T00:59:05.763118+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:53:05.348772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276634,"stat_sum":{"num_bytes":16720704,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":3637,"num_read_kb":20174,"num_write":755,"num_write_kb":2828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:04.988086+0000","last_change":"2024-09-18T00:57:29.327841+0000","last_active":"2024-09-18T00:59:04.988086+0000","last_peered":"2024-09-18T00:59:04.988086+0000","last_clean":"2024-09-18T00:59:04.988086+0000","last_became_active":"2024-09-18T00:57:27.315417+0000","last_became_peered":"2024-09-18T00:57:27.315417+0000","last_unstale":"2024-09-18T00:59:04.988086+0000","last_undegraded":"2024-09-18T00:59:04.988086+0000","last_fullsized":"2024-09-18T00:59:04.988086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:54.335334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"96'261","reported_seq":4732,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.762955+0000","last_change":"2024-09-18T00:57:38.920634+0000","last_active":"2024-09-18T00:59:05.762955+0000","last_peered":"2024-09-18T00:59:05.762955+0000","last_clean":"2024-09-18T00:59:05.762955+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T00:59:05.762955+0000","last_undegraded":"2024-09-18T00:59:05.762955+0000","last_fullsized":"2024-09-18T00:59:05.762955+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:52:12.851801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728301,"stat_sum":{"num_bytes":17969558,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":27,"num_read":5085,"num_read_kb":11854,"num_write":216,"num_write_kb":9054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:04.986989+0000","last_change":"2024-09-18T00:59:01.917810+0000","last_active":"2024-09-18T00:59:04.986989+0000","last_peered":"2024-09-18T00:59:04.986989+0000","last_clean":"2024-09-18T00:59:04.986989+0000","last_became_active":"2024-09-18T00:59:01.916939+0000","last_became_peered":"2024-09-18T00:59:01.916939+0000","last_unstale":"2024-09-18T00:59:04.986989+0000","last_undegraded":"2024-09-18T00:59:04.986989+0000","last_fullsized":"2024-09-18T00:59:04.986989+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:19:05.860921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"88'189","reported_seq":2083,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.347767+0000","last_change":"2024-09-18T00:57:39.056519+0000","last_active":"2024-09-18T00:59:05.347767+0000","last_peered":"2024-09-18T00:59:05.347767+0000","last_clean":"2024-09-18T00:59:05.347767+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T00:59:05.347767+0000","last_undegraded":"2024-09-18T00:59:05.347767+0000","last_fullsized":"2024-09-18T00:59:05.347767+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:39.978483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043106399999999999,"stat_sum":{"num_bytes":30197932,"num_objects":56,"num_object_clones":13,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":27,"num_read":1799,"num_read_kb":22958,"num_write":177,"num_write_kb":15228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":164,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:04.986383+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:59:04.986383+0000","last_peered":"2024-09-18T00:59:04.986383+0000","last_clean":"2024-09-18T00:59:04.986383+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:59:04.986383+0000","last_undegraded":"2024-09-18T00:59:04.986383+0000","last_fullsized":"2024-09-18T00:59:04.986383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"96'388","reported_seq":2174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.738358+0000","last_change":"2024-09-18T00:58:58.965665+0000","last_active":"2024-09-18T00:59:05.738358+0000","last_peered":"2024-09-18T00:59:05.738358+0000","last_clean":"2024-09-18T00:59:05.738358+0000","last_became_active":"2024-09-18T00:58:58.963883+0000","last_became_peered":"2024-09-18T00:58:58.963883+0000","last_unstale":"2024-09-18T00:59:05.738358+0000","last_undegraded":"2024-09-18T00:59:05.738358+0000","last_fullsized":"2024-09-18T00:59:05.738358+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:34:53.462978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000584944,"stat_sum":{"num_bytes":16575667,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":24,"num_read":2806,"num_read_kb":21814,"num_write":645,"num_write_kb":9701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"94'402","reported_seq":3152,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.740054+0000","last_change":"2024-09-18T00:58:58.967999+0000","last_active":"2024-09-18T00:59:05.740054+0000","last_peered":"2024-09-18T00:59:05.740054+0000","last_clean":"2024-09-18T00:59:05.740054+0000","last_became_active":"2024-09-18T00:58:58.965751+0000","last_became_peered":"2024-09-18T00:58:58.965751+0000","last_unstale":"2024-09-18T00:59:05.740054+0000","last_undegraded":"2024-09-18T00:59:05.740054+0000","last_fullsized":"2024-09-18T00:59:05.740054+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:20.708400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056105599999999999,"stat_sum":{"num_bytes":45243390,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":3710,"num_read_kb":13224,"num_write":511,"num_write_kb":20738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":59707869,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:04.991094+0000","last_change":"2024-09-18T00:59:01.926313+0000","last_active":"2024-09-18T00:59:04.991094+0000","last_peered":"2024-09-18T00:59:04.991094+0000","last_clean":"2024-09-18T00:59:04.991094+0000","last_became_active":"2024-09-18T00:59:01.917338+0000","last_became_peered":"2024-09-18T00:59:01.917338+0000","last_unstale":"2024-09-18T00:59:04.991094+0000","last_undegraded":"2024-09-18T00:59:04.991094+0000","last_fullsized":"2024-09-18T00:59:04.991094+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:05:06.044579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'330","reported_seq":5429,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.350575+0000","last_change":"2024-09-18T00:57:38.907753+0000","last_active":"2024-09-18T00:59:05.350575+0000","last_peered":"2024-09-18T00:59:05.350575+0000","last_clean":"2024-09-18T00:59:05.350575+0000","last_became_active":"2024-09-18T00:57:36.907660+0000","last_became_peered":"2024-09-18T00:57:36.907660+0000","last_unstale":"2024-09-18T00:59:05.350575+0000","last_undegraded":"2024-09-18T00:59:05.350575+0000","last_fullsized":"2024-09-18T00:59:05.350575+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:01.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090282999999999997,"stat_sum":{"num_bytes":38518023,"num_objects":78,"num_object_clones":21,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":35,"num_read":5530,"num_read_kb":21777,"num_write":368,"num_write_kb":7502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-18T00:59:03.053651+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:59:03.053651+0000","last_peered":"2024-09-18T00:59:03.053651+0000","last_clean":"2024-09-18T00:59:03.053651+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:59:03.053651+0000","last_undegraded":"2024-09-18T00:59:03.053651+0000","last_fullsized":"2024-09-18T00:59:03.053651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"96'313","reported_seq":3711,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.735142+0000","last_change":"2024-09-18T00:58:58.966926+0000","last_active":"2024-09-18T00:59:05.735142+0000","last_peered":"2024-09-18T00:59:05.735142+0000","last_clean":"2024-09-18T00:59:05.735142+0000","last_became_active":"2024-09-18T00:58:58.964827+0000","last_became_peered":"2024-09-18T00:58:58.964827+0000","last_unstale":"2024-09-18T00:59:05.735142+0000","last_undegraded":"2024-09-18T00:59:05.735142+0000","last_fullsized":"2024-09-18T00:59:05.735142+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:55.759055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045206800000000002,"stat_sum":{"num_bytes":19784472,"num_objects":66,"num_object_clones":14,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":26,"num_read":4152,"num_read_kb":20914,"num_write":299,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":9938529,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:04.986399+0000","last_change":"2024-09-18T00:59:01.918199+0000","last_active":"2024-09-18T00:59:04.986399+0000","last_peered":"2024-09-18T00:59:04.986399+0000","last_clean":"2024-09-18T00:59:04.986399+0000","last_became_active":"2024-09-18T00:59:01.916643+0000","last_became_peered":"2024-09-18T00:59:01.916643+0000","last_unstale":"2024-09-18T00:59:04.986399+0000","last_undegraded":"2024-09-18T00:59:04.986399+0000","last_fullsized":"2024-09-18T00:59:04.986399+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:22.282208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'548","reported_seq":2758,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.763014+0000","last_change":"2024-09-18T00:58:58.966801+0000","last_active":"2024-09-18T00:59:05.763014+0000","last_peered":"2024-09-18T00:59:05.763014+0000","last_clean":"2024-09-18T00:59:05.763014+0000","last_became_active":"2024-09-18T00:58:58.963566+0000","last_became_peered":"2024-09-18T00:58:58.963566+0000","last_unstale":"2024-09-18T00:59:05.763014+0000","last_undegraded":"2024-09-18T00:59:05.763014+0000","last_fullsized":"2024-09-18T00:59:05.763014+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:40:06.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064609,"stat_sum":{"num_bytes":22358913,"num_objects":59,"num_object_clones":8,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":30,"num_read":3379,"num_read_kb":10055,"num_write":910,"num_write_kb":38221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":8652757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":184,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:04.988061+0000","last_change":"2024-09-18T00:59:01.921709+0000","last_active":"2024-09-18T00:59:04.988061+0000","last_peered":"2024-09-18T00:59:04.988061+0000","last_clean":"2024-09-18T00:59:04.988061+0000","last_became_active":"2024-09-18T00:59:01.916229+0000","last_became_peered":"2024-09-18T00:59:01.916229+0000","last_unstale":"2024-09-18T00:59:04.988061+0000","last_undegraded":"2024-09-18T00:59:04.988061+0000","last_fullsized":"2024-09-18T00:59:04.988061+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:26.611604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":316,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:04.981274+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:59:04.981274+0000","last_peered":"2024-09-18T00:59:04.981274+0000","last_clean":"2024-09-18T00:59:04.981274+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:59:04.981274+0000","last_undegraded":"2024-09-18T00:59:04.981274+0000","last_fullsized":"2024-09-18T00:59:04.981274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"88'253","reported_seq":3145,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.496416+0000","last_change":"2024-09-18T00:57:38.919534+0000","last_active":"2024-09-18T00:59:05.496416+0000","last_peered":"2024-09-18T00:59:05.496416+0000","last_clean":"2024-09-18T00:59:05.496416+0000","last_became_active":"2024-09-18T00:57:36.908955+0000","last_became_peered":"2024-09-18T00:57:36.908955+0000","last_unstale":"2024-09-18T00:59:05.496416+0000","last_undegraded":"2024-09-18T00:59:05.496416+0000","last_fullsized":"2024-09-18T00:59:05.496416+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:29:38.897577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077760500000000001,"stat_sum":{"num_bytes":41496428,"num_objects":64,"num_object_clones":14,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":29,"num_read":2889,"num_read_kb":36956,"num_write":308,"num_write_kb":34796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"94'234","reported_seq":1723,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.740329+0000","last_change":"2024-09-18T00:59:04.995345+0000","last_active":"2024-09-18T00:59:05.740329+0000","last_peered":"2024-09-18T00:59:05.740329+0000","last_clean":"2024-09-18T00:59:05.740329+0000","last_became_active":"2024-09-18T00:59:04.993568+0000","last_became_peered":"2024-09-18T00:59:04.993568+0000","last_unstale":"2024-09-18T00:59:05.740329+0000","last_undegraded":"2024-09-18T00:59:05.740329+0000","last_fullsized":"2024-09-18T00:59:05.740329+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:27.659824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061570699999999995,"stat_sum":{"num_bytes":20267223,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":1596,"num_read_kb":6211,"num_write":206,"num_write_kb":17217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":8365403,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'225","reported_seq":2148,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.348738+0000","last_change":"2024-09-18T00:59:04.993223+0000","last_active":"2024-09-18T00:59:05.348738+0000","last_peered":"2024-09-18T00:59:05.348738+0000","last_clean":"2024-09-18T00:59:05.348738+0000","last_became_active":"2024-09-18T00:59:04.992636+0000","last_became_peered":"2024-09-18T00:59:04.992636+0000","last_unstale":"2024-09-18T00:59:05.348738+0000","last_undegraded":"2024-09-18T00:59:05.348738+0000","last_fullsized":"2024-09-18T00:59:05.348738+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:32.995943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097047300000000002,"stat_sum":{"num_bytes":16167719,"num_objects":57,"num_object_clones":8,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":1684,"num_read_kb":20074,"num_write":190,"num_write_kb":10540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":1719706,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"92'329","reported_seq":2460,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-18T00:59:03.204422+0000","last_change":"2024-09-18T00:57:38.899763+0000","last_active":"2024-09-18T00:59:03.204422+0000","last_peered":"2024-09-18T00:59:03.204422+0000","last_clean":"2024-09-18T00:59:03.204422+0000","last_became_active":"2024-09-18T00:57:36.905869+0000","last_became_peered":"2024-09-18T00:57:36.905869+0000","last_unstale":"2024-09-18T00:59:03.204422+0000","last_undegraded":"2024-09-18T00:59:03.204422+0000","last_fullsized":"2024-09-18T00:59:03.204422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:25:34.617332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097615600000000005,"stat_sum":{"num_bytes":20320541,"num_objects":71,"num_object_clones":21,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":28,"num_read":2718,"num_read_kb":28318,"num_write":346,"num_write_kb":6078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"92'363","reported_seq":3783,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.730322+0000","last_change":"2024-09-18T00:57:38.909165+0000","last_active":"2024-09-18T00:59:05.730322+0000","last_peered":"2024-09-18T00:59:05.730322+0000","last_clean":"2024-09-18T00:59:05.730322+0000","last_became_active":"2024-09-18T00:57:36.908288+0000","last_became_peered":"2024-09-18T00:57:36.908288+0000","last_unstale":"2024-09-18T00:59:05.730322+0000","last_undegraded":"2024-09-18T00:59:05.730322+0000","last_fullsized":"2024-09-18T00:59:05.730322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:24.568583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044375399999999998,"stat_sum":{"num_bytes":34475138,"num_objects":62,"num_object_clones":18,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":22,"num_read":4186,"num_read_kb":28598,"num_write":479,"num_write_kb":16891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"94'612","reported_seq":3718,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.343643+0000","last_change":"2024-09-18T00:59:04.984309+0000","last_active":"2024-09-18T00:59:05.343643+0000","last_peered":"2024-09-18T00:59:05.343643+0000","last_clean":"2024-09-18T00:59:05.343643+0000","last_became_active":"2024-09-18T00:59:04.983302+0000","last_became_peered":"2024-09-18T00:59:04.983302+0000","last_unstale":"2024-09-18T00:59:05.343643+0000","last_undegraded":"2024-09-18T00:59:05.343643+0000","last_fullsized":"2024-09-18T00:59:05.343643+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:10.919126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023201229999999999,"stat_sum":{"num_bytes":46919400,"num_objects":73,"num_object_clones":17,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":31,"num_read":5925,"num_read_kb":30598,"num_write":841,"num_write_kb":14475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13370804,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"96'495","reported_seq":1895,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.760850+0000","last_change":"2024-09-18T00:58:58.968017+0000","last_active":"2024-09-18T00:59:05.760850+0000","last_peered":"2024-09-18T00:59:05.760850+0000","last_clean":"2024-09-18T00:59:05.760850+0000","last_became_active":"2024-09-18T00:58:58.964808+0000","last_became_peered":"2024-09-18T00:58:58.964808+0000","last_unstale":"2024-09-18T00:59:05.760850+0000","last_undegraded":"2024-09-18T00:59:05.760850+0000","last_fullsized":"2024-09-18T00:59:05.760850+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:27.235079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090629000000000003,"stat_sum":{"num_bytes":19350653,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":23,"num_read":2788,"num_read_kb":13211,"num_write":731,"num_write_kb":10238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":9652505,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'315","reported_seq":2517,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.743620+0000","last_change":"2024-09-18T00:57:38.909597+0000","last_active":"2024-09-18T00:59:05.743620+0000","last_peered":"2024-09-18T00:59:05.743620+0000","last_clean":"2024-09-18T00:59:05.743620+0000","last_became_active":"2024-09-18T00:57:36.905462+0000","last_became_peered":"2024-09-18T00:57:36.905462+0000","last_unstale":"2024-09-18T00:59:05.743620+0000","last_undegraded":"2024-09-18T00:59:05.743620+0000","last_fullsized":"2024-09-18T00:59:05.743620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:13:39.484194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081831799999999995,"stat_sum":{"num_bytes":26138767,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":3527,"num_read_kb":22810,"num_write":346,"num_write_kb":7877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"92'277","reported_seq":3246,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.745020+0000","last_change":"2024-09-18T00:57:38.900906+0000","last_active":"2024-09-18T00:59:05.745020+0000","last_peered":"2024-09-18T00:59:05.745020+0000","last_clean":"2024-09-18T00:59:05.745020+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T00:59:05.745020+0000","last_undegraded":"2024-09-18T00:59:05.745020+0000","last_fullsized":"2024-09-18T00:59:05.745020+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:52:40.333034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069386899999999995,"stat_sum":{"num_bytes":39366539,"num_objects":73,"num_object_clones":15,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":32,"num_read":3538,"num_read_kb":38775,"num_write":237,"num_write_kb":25045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"94'188","reported_seq":2087,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.342963+0000","last_change":"2024-09-18T00:59:04.992636+0000","last_active":"2024-09-18T00:59:05.342963+0000","last_peered":"2024-09-18T00:59:05.342963+0000","last_clean":"2024-09-18T00:59:05.342963+0000","last_became_active":"2024-09-18T00:59:04.991454+0000","last_became_peered":"2024-09-18T00:59:04.991454+0000","last_unstale":"2024-09-18T00:59:05.342963+0000","last_undegraded":"2024-09-18T00:59:05.342963+0000","last_fullsized":"2024-09-18T00:59:05.342963+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:08:06.298691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071487799999999995,"stat_sum":{"num_bytes":21955251,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":1359,"num_read_kb":12525,"num_write":151,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":13441939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"94'317","reported_seq":3846,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.740354+0000","last_change":"2024-09-18T00:59:04.995178+0000","last_active":"2024-09-18T00:59:05.740354+0000","last_peered":"2024-09-18T00:59:05.740354+0000","last_clean":"2024-09-18T00:59:05.740354+0000","last_became_active":"2024-09-18T00:59:04.993698+0000","last_became_peered":"2024-09-18T00:59:04.993698+0000","last_unstale":"2024-09-18T00:59:05.740354+0000","last_undegraded":"2024-09-18T00:59:05.740354+0000","last_fullsized":"2024-09-18T00:59:05.740354+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:35:24.582484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055104700000000004,"stat_sum":{"num_bytes":29967785,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":24,"num_read":4275,"num_read_kb":15142,"num_write":340,"num_write_kb":9756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":8283018,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"96'358","reported_seq":1758,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-18T00:59:05.736893+0000","last_change":"2024-09-18T00:59:04.993330+0000","last_active":"2024-09-18T00:59:05.736893+0000","last_peered":"2024-09-18T00:59:05.736893+0000","last_clean":"2024-09-18T00:59:05.736893+0000","last_became_active":"2024-09-18T00:59:04.992349+0000","last_became_peered":"2024-09-18T00:59:04.992349+0000","last_unstale":"2024-09-18T00:59:05.736893+0000","last_undegraded":"2024-09-18T00:59:05.736893+0000","last_fullsized":"2024-09-18T00:59:05.736893+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:57:18.606627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070399799999999995,"stat_sum":{"num_bytes":10448230,"num_objects":45,"num_object_clones":9,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":18,"num_read":2675,"num_read_kb":8354,"num_write":443,"num_write_kb":3829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2320662,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":939459381,"num_objects":1876,"num_object_clones":420,"num_object_copies":3752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1876,"num_whiteouts":837,"num_read":145067,"num_read_kb":684742,"num_write":14163,"num_write_kb":475659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":207217810,"num_keys_recovered":402,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":608071680,"data_stored":741763715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11560,"ondisk_log_size":11560,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91940,"kb_used_data":64916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94279900,"statfs":{"total":96636764160,"available":96542617600,"internally_reserved":0,"allocated":66473984,"data_stored":82096709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106488,"kb_used_data":79456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94265352,"statfs":{"total":96636764160,"available":96527720448,"internally_reserved":0,"allocated":81362944,"data_stored":103759756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509466,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132260,"kb_used_data":101068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94239580,"statfs":{"total":96636764160,"available":96501329920,"internally_reserved":0,"allocated":103493632,"data_stored":118289093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":64424509466,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":64424509466,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127796,"kb_used_data":100764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94244044,"statfs":{"total":96636764160,"available":96505901056,"internally_reserved":0,"allocated":103182336,"data_stored":124578672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":151444,"kb_used_data":98052,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94220396,"statfs":{"total":96636764160,"available":96481685504,"internally_reserved":0,"allocated":100405248,"data_stored":120365449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80892,"kb_used_data":41380,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94290948,"statfs":{"total":96636764160,"available":96553930752,"internally_reserved":0,"allocated":42373120,"data_stored":49965805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":99274752,"data_stored":119845812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":101453824,"data_stored":123468648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52674560,"data_stored":68201235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":63754240,"data_stored":76508402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102363136,"data_stored":117769456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":79638528,"data_stored":102649751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65339392,"data_stored":81577053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":43573248,"data_stored":51743358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:59:06.715 INFO:tasks.thrashosds.thrasher:Setting 2.1 to [5, 3] 2024-09-18T00:59:06.715 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.1', '5', '3'] 2024-09-18T00:59:06.715 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.1 5 3 2024-09-18T00:59:06.729 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:06.729 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:06.752 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:06.797 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:06.793+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:06.870 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:06.865+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:06.892 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:06.903 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:06.904 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:06.918 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:06.963 INFO:teuthology.orchestra.run.smithi042.stdout:3727 write 0xe653e9a thru 0xe658d61 (0x4ec8 bytes) 2024-09-18T00:59:06.967 INFO:teuthology.orchestra.run.smithi042.stdout:3728 read 0x52f251e thru 0x52f6b6d (0x4650 bytes) 2024-09-18T00:59:06.967 INFO:teuthology.orchestra.run.smithi042.stdout:3729 write 0xbb7b3da thru 0xbb84512 (0x9139 bytes) 2024-09-18T00:59:06.971 INFO:teuthology.orchestra.run.smithi042.stdout:3730 punch from 0x8ec6ebd to 0x8eca9dd, (0x3b20 bytes) 2024-09-18T00:59:06.971 INFO:teuthology.orchestra.run.smithi042.stdout:3731 write 0x8a018ed thru 0x8a11713 (0xfe27 bytes) 2024-09-18T00:59:06.971 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:06.965+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:06.974 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:06.974 INFO:teuthology.orchestra.run.smithi042.stdout:3734 writesame 0xba302cf thru 0xba35c32 data_size 0x6(0x5964 bytes) 2024-09-18T00:59:06.979 INFO:teuthology.orchestra.run.smithi042.stdout:3736 write 0x65aa5d9 thru 0x65b2fee (0x8a16 bytes) 2024-09-18T00:59:06.983 INFO:teuthology.orchestra.run.smithi042.stdout:3737 read 0x1e39a03 thru 0x1e43e23 (0xa421 bytes) 2024-09-18T00:59:06.984 INFO:teuthology.orchestra.run.smithi042.stdout:3738 writesame 0x3f11bf0 thru 0x3f1ef64 data_size 0x1(0xd375 bytes) 2024-09-18T00:59:07.072 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:07.065+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:07.073 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:07.073 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:07.091 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:07.109 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:07.110 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:07.127 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:07.128 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:07.141 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:07.141 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:07.147 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:07.148 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:07.161 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:07.165 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:07.165 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:07.185 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:07.202 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:07.203 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:07.209 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:07.205+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:07.273 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:07.269+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:07.310 INFO:teuthology.orchestra.run.smithi042.stdout:3739 trunc from 0xe658d62 to 0xcbf44ff 2024-09-18T00:59:07.311 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:07.311 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:07.311 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:07.311 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:07.326 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:07.328 INFO:teuthology.orchestra.run.smithi042.stdout:3741 trunc from 0xcbf44ff to 0x94824fd 2024-09-18T00:59:07.349 INFO:teuthology.orchestra.run.smithi042.stdout:3742 read 0x2a29546 thru 0x2a2a93b (0x13f6 bytes) 2024-09-18T00:59:07.349 INFO:teuthology.orchestra.run.smithi042.stdout:3743 write 0x1eb5a0e thru 0x1eb91b7 (0x37aa bytes) 2024-09-18T00:59:07.351 INFO:teuthology.orchestra.run.smithi042.stdout:3744 punch from 0x104da91 to 0x105692b, (0x8e9a bytes) 2024-09-18T00:59:07.351 INFO:teuthology.orchestra.run.smithi042.stdout:3745 clone 32 order 24 su 4194304 sc 3 2024-09-18T00:59:07.413 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:07.409+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:07.476 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:07.473+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:07.491 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:07.492 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:07.492 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:07.492 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:07.492 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:07.506 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:07.577 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:07.569+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:07.678 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:07.673+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:07.679 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:07.679 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.267981875999993, 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.680 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:07.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T00:59:07.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.685 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:26797 3.0 3:03a9420d:::rbd_data.117c4c7db1a0.0000000000000071:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.489555076999999, 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073789632999999993, 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.686 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 26797 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187638+0000", 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187640+0000", 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187653+0000", 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2357e-05 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187684+0000", 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1582999999999999e-05 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187699+0000", 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5003e-05 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.187806+0000", 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106776 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.188026+0000", 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021975799999999999 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7", 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.201663+0000", 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013637768 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261226+0000", 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059562916 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261261+0000", 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4360000000000003e-05 2024-09-18T00:59:07.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261354+0000", 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3041999999999995e-05 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:07.261427+0000", 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3553000000000005e-05 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64358 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.087421757999998, 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054261125, 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64358 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589771+0000", 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589772+0000", 2024-09-18T00:59:07.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1170000000000001e-06 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589778+0000", 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8429999999999998e-06 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589799+0000", 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0727e-05 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589813+0000", 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4403000000000001e-05 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644021+0000", 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054207784000000002 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644032+0000", 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1251e-05 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.695 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64359 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.087359792000001, 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286586999999997, 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64359 2024-09-18T00:59:07.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589833+0000", 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589834+0000", 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589841+0000", 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.956e-06 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589864+0000", 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3124000000000001e-05 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589879+0000", 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.486e-05 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644099+0000", 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054219429 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644120+0000", 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1004000000000001e-05 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.699 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64360 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.087295597999997, 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054290699999999997, 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.700 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64360 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589897+0000", 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589898+0000", 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3600000000000002e-07 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589903+0000", 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4669999999999998e-06 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589913+0000", 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0463e-05 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589921+0000", 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7959999999999996e-06 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644173+0000", 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054251903999999997 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644188+0000", 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5133999999999999e-05 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64361 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.087262119000002, 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054320101000000003, 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64361 2024-09-18T00:59:07.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589931+0000", 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5000000000000002e-07 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589935+0000", 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2799999999999999e-06 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589944+0000", 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589954+0000", 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0036e-05 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644240+0000", 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054286273000000003 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644251+0000", 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0392e-05 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.707 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64362 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.087228973000002, 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054340196, 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.708 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64362 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589964+0000", 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589978+0000", 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3570000000000001e-05 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.589991+0000", 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3196999999999999e-05 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590000+0000", 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2800000000000003e-06 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644294+0000", 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054294873 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644304+0000", 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6560000000000004e-06 2024-09-18T00:59:07.711 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:64363 3.1e 3.e9d1b0fe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.087182943000002, 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054356660000000001, 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.712 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 64363 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590010+0000", 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590011+0000", 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7400000000000003e-07 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590014+0000", 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590024+0000", 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5670000000000001e-06 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.590031+0000", 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3830000000000003e-06 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644354+0000", 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054323429999999999 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:25.644367+0000", 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.217e-05 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.715 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108735 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 278528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208260+0000", 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4689327859999999, 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054537304000000002, 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108735 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.716 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208260+0000", 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208260+0000", 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208260+0000", 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.29e-07 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208261+0000", 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.217e-06 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208265+0000", 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.18e-06 2024-09-18T00:59:07.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208268+0000", 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8660000000000002e-06 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.262299+0000", 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054031719999999998 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.262642+0000", 2024-09-18T00:59:07.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034325300000000002 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.262797+0000", 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015483900000000001 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108736 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208271+0000", 2024-09-18T00:59:07.720 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4689220409999999, 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055233697999999998, 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108736 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208271+0000", 2024-09-18T00:59:07.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208271+0000", 2024-09-18T00:59:07.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208271+0000", 2024-09-18T00:59:07.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.28e-07 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208272+0000", 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1689999999999999e-06 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208275+0000", 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.27e-06 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208278+0000", 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6680000000000001e-06 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.262831+0000", 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054552490000000002 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.263282+0000", 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045121899999999999 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.263504+0000", 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000222654 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108737 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 319488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208282+0000", 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4689107879999996, 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055583957000000003, 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.725 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108737 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208282+0000", 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208282+0000", 2024-09-18T00:59:07.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208282+0000", 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208284+0000", 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0320000000000002e-06 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208291+0000", 2024-09-18T00:59:07.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.331e-06 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208294+0000", 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.213e-06 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.263556+0000", 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055262520000000002 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.263643+0000", 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6390000000000005e-05 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.263866+0000", 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022315900000000001 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108738 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208298+0000", 2024-09-18T00:59:07.729 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4688948140000004, 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055750797999999997, 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108738 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208298+0000", 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208298+0000", 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208298+0000", 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7300000000000002e-07 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208300+0000", 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.345e-06 2024-09-18T00:59:07.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208305+0000", 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5520000000000004e-06 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208310+0000", 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3780000000000003e-06 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.263901+0000", 2024-09-18T00:59:07.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055591079000000002 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.263957+0000", 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6184999999999998e-05 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.264049+0000", 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1985999999999996e-05 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108739 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208316+0000", 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4688770839999998, 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057114401000000002, 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108739 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208316+0000", 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208316+0000", 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208316+0000", 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.41e-07 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208318+0000", 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9870000000000002e-06 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208325+0000", 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8260000000000004e-06 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208330+0000", 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1309999999999997e-06 2024-09-18T00:59:07.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.264867+0000", 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056536771999999999 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.264950+0000", 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3332999999999996e-05 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.265430+0000", 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048001100000000002 2024-09-18T00:59:07.737 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108740 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 389120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208336+0000", 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4688563639999996, 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057329984, 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108740 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208336+0000", 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208336+0000", 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208337+0000", 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3700000000000001e-07 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208340+0000", 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7889999999999999e-06 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208343+0000", 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7680000000000002e-06 2024-09-18T00:59:07.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208348+0000", 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3649999999999997e-06 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.265463+0000", 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057115351000000002 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.265547+0000", 2024-09-18T00:59:07.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3750000000000003e-05 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.265666+0000", 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011962400000000001 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108741 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208373+0000", 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4688201699999999, 2024-09-18T00:59:07.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057763244999999998, 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108741 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208373+0000", 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208373+0000", 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208373+0000", 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5600000000000005e-07 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208375+0000", 2024-09-18T00:59:07.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.601e-06 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208379+0000", 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3359999999999997e-06 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208384+0000", 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7910000000000004e-06 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.265698+0000", 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057313887000000001 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.266001+0000", 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030335900000000001 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.266136+0000", 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013461500000000001 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.746 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108742 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 430080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208391+0000", 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4688017149999997, 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057987077999999997, 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.747 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108742 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208391+0000", 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208391+0000", 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208392+0000", 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9800000000000003e-07 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208394+0000", 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3920000000000001e-06 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208401+0000", 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3550000000000001e-06 2024-09-18T00:59:07.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208406+0000", 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3210000000000004e-06 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.266169+0000", 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057762920000000002 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.266248+0000", 2024-09-18T00:59:07.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8817000000000001e-05 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.266378+0000", 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013057499999999999 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108743 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 442368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208412+0000", 2024-09-18T00:59:07.751 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4687808499999999, 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058560014000000001, 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108743 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208412+0000", 2024-09-18T00:59:07.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208412+0000", 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208412+0000", 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8000000000000001e-07 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208414+0000", 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.849e-06 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208420+0000", 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2949999999999999e-06 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208426+0000", 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6239999999999999e-06 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.266414+0000", 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057988115 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.266842+0000", 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042759599999999998 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.266972+0000", 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013015499999999999 2024-09-18T00:59:07.755 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108744 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208434+0000", 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.468759028, 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058784116999999997, 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.756 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108744 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208434+0000", 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208434+0000", 2024-09-18T00:59:07.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208434+0000", 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.84e-07 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208436+0000", 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.243e-06 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208443+0000", 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1750000000000002e-06 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208446+0000", 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4809999999999998e-06 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.267005+0000", 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058558564 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.267091+0000", 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6451000000000003e-05 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.267218+0000", 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012681899999999999 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.760 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108745 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208452+0000", 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4687406960000002, 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060159016000000003, 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108745 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208452+0000", 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208452+0000", 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208452+0000", 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.84e-07 2024-09-18T00:59:07.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208460+0000", 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4000000000000003e-06 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208467+0000", 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6789999999999999e-06 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208472+0000", 2024-09-18T00:59:07.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1170000000000003e-06 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.268326+0000", 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059853946999999998 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.268437+0000", 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011124900000000001 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.268611+0000", 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017424000000000001 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108746 3.17 3:ee485403:::rbd_data.117c8e0d79e4.0000000000000035:20 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.208478+0000", 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4687144520000004, 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060354761, 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:07.765 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108746 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208478+0000", 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:07.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208478+0000", 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208479+0000", 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208481+0000", 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0040000000000002e-06 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208487+0000", 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4799999999999998e-06 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.208492+0000", 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1540000000000001e-06 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.268642+0000", 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060150092000000002 2024-09-18T00:59:07.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.268718+0000", 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5438999999999994e-05 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.268833+0000", 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115244 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.769 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:07.770 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:07.770 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:07.771 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:07.778 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:07.777+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:07.880 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:07.873+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:07.940 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:07.940 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:07.941 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:07.941 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:07.955 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:07.997 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:07.993+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:set 2.1 pg_upmap mapping to [5,3] 2024-09-18T00:59:08.021 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:59:08.081 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:08.077+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:08.129 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:08.130 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:08.139 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone31 2024-09-18T00:59:08.148 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:08.190 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:08.185+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:08.283 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:08.281+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:08.358 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.268865610999995, 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.877970660000003, 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.363 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T00:59:08.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T00:59:08.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T00:59:08.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T00:59:08.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T00:59:08.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78518 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1183744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.526439831000001, 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054532499999999998, 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.368 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78518 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831040+0000", 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3099999999999997e-07 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831042+0000", 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2879999999999999e-06 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831050+0000", 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.464e-06 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831055+0000", 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0819999999999998e-06 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885354+0000", 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298739999999998 2024-09-18T00:59:08.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885428+0000", 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4567999999999995e-05 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885572+0000", 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014372699999999999 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78519 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1204224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.526417982000002, 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054707261, 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78519 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.373 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831062+0000", 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:59:08.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831064+0000", 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0360000000000001e-06 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831069+0000", 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4149999999999998e-06 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831075+0000", 2024-09-18T00:59:08.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2249999999999997e-06 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885602+0000", 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054526893 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885673+0000", 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0865000000000007e-05 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885768+0000", 2024-09-18T00:59:08.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5422999999999998e-05 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78520 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1224704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.526397526, 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054906695999999998, 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.377 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78520 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831084+0000", 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2960000000000001e-06 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831091+0000", 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:59:08.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831095+0000", 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.792e-06 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885802+0000", 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054706795000000003 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885884+0000", 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2297000000000002e-05 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885988+0000", 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010393199999999999 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78521 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:08.381 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.526352653, 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055066616999999998, 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78521 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831127+0000", 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.99e-07 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831130+0000", 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6819999999999999e-06 2024-09-18T00:59:08.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831137+0000", 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3849999999999996e-06 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831145+0000", 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7360000000000001e-06 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886020+0000", 2024-09-18T00:59:08.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054874700999999998 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886096+0000", 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6234000000000002e-05 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886193+0000", 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7280000000000004e-05 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78522 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.526326892, 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055137938999999997, 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78522 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831153+0000", 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.37e-07 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831155+0000", 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4420000000000001e-06 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831161+0000", 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831166+0000", 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8749999999999999e-06 2024-09-18T00:59:08.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886279+0000", 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055112337999999997 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886290+0000", 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1527e-05 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78523 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.526306212000001, 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055171425000000003, 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78523 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0900000000000002e-07 2024-09-18T00:59:08.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831176+0000", 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4830000000000002e-06 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831183+0000", 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0679999999999999e-06 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831189+0000", 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886331+0000", 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055142669999999998 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886344+0000", 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2918999999999999e-05 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.393 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78524 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.526284096000001, 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055194430000000003, 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.394 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78524 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831196+0000", 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1200000000000003e-07 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831198+0000", 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831204+0000", 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2740000000000004e-06 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831209+0000", 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0900000000000004e-06 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886381+0000", 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055172142 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886389+0000", 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2150000000000001e-06 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.397 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78525 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.526263582999999, 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055206535000000001, 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78525 2024-09-18T00:59:08.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832358+0000", 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001142531 2024-09-18T00:59:08.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832364+0000", 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832375+0000", 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1381e-05 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832382+0000", 2024-09-18T00:59:08.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9800000000000001e-06 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886415+0000", 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032678000000001 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886422+0000", 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4449999999999999e-06 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108130 3.1 3:83363b4a:::rbd_data.117c8e0d79e4.000000000000002d:20 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.1635485130000003, 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054351575999999999, 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108130 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:08.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1100000000000001e-07 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193933+0000", 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3690000000000001e-06 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193940+0000", 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1460000000000001e-06 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193944+0000", 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9230000000000001e-06 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.245560+0000", 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051615019999999998 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.245633+0000", 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3490000000000003e-05 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.248282+0000", 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026492170000000002 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108393 3.1 3:86b5547c:::rbd_data.117c8e0d79e4.0000000000000037:20 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.1604569580000001, 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054339392, 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.406 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108393 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5499999999999999e-07 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197024+0000", 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.308e-06 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197029+0000", 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197032+0000", 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.994e-06 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251285+0000", 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054253006999999999 2024-09-18T00:59:08.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251322+0000", 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7104000000000003e-05 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251362+0000", 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9941000000000003e-05 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108394 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.1603945519999996, 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054381878000000002, 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108394 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.411 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5400000000000001e-07 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197088+0000", 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8830000000000002e-06 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197096+0000", 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8560000000000007e-06 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197101+0000", 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251388+0000", 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054287588999999997 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251427+0000", 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8665999999999999e-05 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251466+0000", 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9369000000000002e-05 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108395 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.160369137, 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054457976999999998, 2024-09-18T00:59:08.415 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108395 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197111+0000", 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6000000000000004e-07 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197113+0000", 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4119999999999999e-06 2024-09-18T00:59:08.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197119+0000", 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4160000000000001e-06 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197125+0000", 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8320000000000002e-06 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251492+0000", 2024-09-18T00:59:08.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054366997 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251529+0000", 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6563000000000002e-05 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251568+0000", 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9196999999999998e-05 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108396 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.1603460659999998, 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054537624, 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108396 2024-09-18T00:59:08.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6199999999999999e-07 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197135+0000", 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197139+0000", 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1749999999999998e-06 2024-09-18T00:59:08.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197143+0000", 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1350000000000001e-06 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251594+0000", 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054451160999999998 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251632+0000", 2024-09-18T00:59:08.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7840999999999999e-05 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251671+0000", 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9169999999999999e-05 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108413 3.1 3:8453e3d7:::rbd_data.117c8e0d79e4.0000000000000022:20 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.1599020580000001, 2024-09-18T00:59:08.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054401959, 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108413 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197578+0000", 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7199999999999999e-07 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197581+0000", 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8439999999999998e-06 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197587+0000", 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1759999999999998e-06 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197592+0000", 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4310000000000002e-06 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251903+0000", 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054310418999999999 2024-09-18T00:59:08.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251939+0000", 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6310999999999998e-05 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251979+0000", 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0206000000000001e-05 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108428 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.1597076199999998, 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054482228000000001, 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108428 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9900000000000001e-07 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197774+0000", 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.373e-06 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197781+0000", 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3130000000000004e-06 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197786+0000", 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9980000000000003e-06 2024-09-18T00:59:08.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252117+0000", 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054331329999999997 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252178+0000", 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0952000000000002e-05 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252254+0000", 2024-09-18T00:59:08.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5863000000000002e-05 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108429 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.1596866710000002, 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054624675999999997, 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.433 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108429 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197793+0000", 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0600000000000001e-07 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197795+0000", 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.458e-06 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197802+0000", 2024-09-18T00:59:08.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.691e-06 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197807+0000", 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2249999999999999e-06 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252283+0000", 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054476138 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252348+0000", 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4783999999999995e-05 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252417+0000", 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8973999999999994e-05 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108430 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:08.437 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.15966448, 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054759302000000003, 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108430 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.58e-07 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197818+0000", 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5600000000000001e-06 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197824+0000", 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.511e-06 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197829+0000", 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1619999999999999e-06 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252446+0000", 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054616988999999998 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252513+0000", 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6558999999999998e-05 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252574+0000", 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1062999999999995e-05 2024-09-18T00:59:08.441 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108540 3.1 3:83363b4a:::rbd_data.117c8e0d79e4.000000000000002d:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.1522851010000004, 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056511851000000002, 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:08.442 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108540 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.01e-07 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205197+0000", 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205204+0000", 2024-09-18T00:59:08.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4610000000000002e-06 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205210+0000", 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5140000000000001e-06 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.256876+0000", 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051666493000000001 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.256915+0000", 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8724999999999999e-05 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.261706+0000", 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004791086 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:08.446 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:08.448 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:08.448 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:08.441+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:08.549 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:08.545+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:08.596 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-18T00:59:08.619 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:08.619 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:08.619 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:08.619 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:08.642 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:08.685 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:08.681+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:08.756 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:08.745+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:08.851 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:08.849+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:08.866 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:08.867 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:08.887 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:08.977 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:08.973+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:09.011 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:59:09.052 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:09.045+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:09.112 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4041 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.108271259999995, 2024-09-18T00:59:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037088000000000003, 2024-09-18T00:59:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4041 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:09.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003005+0000", 2024-09-18T00:59:09.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.187e-06 2024-09-18T00:59:09.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003011+0000", 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003029+0000", 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8026999999999999e-05 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003043+0000", 2024-09-18T00:59:09.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2024-09-18T00:59:09.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039484+0000", 2024-09-18T00:59:09.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036440895000000001 2024-09-18T00:59:09.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039722+0000", 2024-09-18T00:59:09.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000238507 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040092+0000", 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000369818 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:09.118 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.108213637000006, 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:59:09.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:59:09.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:59:09.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:59:09.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:59:09.123 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.123 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.123 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.123 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.108155379999999, 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.124 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:59:09.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:59:09.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:59:09.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:59:09.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:59:09.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:59:09.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:59:09.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:59:09.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.128 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.129 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:09.129 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:09.129 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.108096940999999, 2024-09-18T00:59:09.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:59:09.129 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.129 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.129 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:09.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:59:09.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:59:09.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:59:09.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:09.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:59:09.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:59:09.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:59:09.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:59:09.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:59:09.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:59:09.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:59:09.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:59:09.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:59:09.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:59:09.134 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.134 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.135 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.135 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:09.135 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:09.135 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.108035724000004, 2024-09-18T00:59:09.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:59:09.135 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.135 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.135 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:09.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:59:09.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:59:09.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:59:09.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:59:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:59:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.107945309000002, 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:59:09.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:59:09.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:59:09.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:59:09.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:59:09.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:59:09.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.145 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.107874375999998, 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.146 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:09.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:09.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:59:09.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:59:09.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:59:09.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:59:09.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:59:09.150 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.150 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.107814770000005, 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:09.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:59:09.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.107755400000002, 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.157 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.157 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.157 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:59:09.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.157 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:09.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:59:09.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:59:09.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:59:09.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:59:09.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:59:09.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.107693611000002, 2024-09-18T00:59:09.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:59:09.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:59:09.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:59:09.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:59:09.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:59:09.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:59:09.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:59:09.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:59:09.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:59:09.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.167 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:09.167 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:09.167 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.107560788000001, 2024-09-18T00:59:09.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:59:09.167 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.167 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.167 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.167 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.168 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.168 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.168 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:59:09.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.168 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:09.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:09.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:59:09.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:59:09.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.552343311000001, 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.551843187000003, 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.180 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:09.184 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.551812678999994, 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T00:59:09.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:09.188 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.551788013999996, 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:59:09.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:09.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T00:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.551759481999994, 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.193 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T00:59:09.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T00:59:09.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.551426191000004, 2024-09-18T00:59:09.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T00:59:09.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T00:59:09.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.551384339999998, 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.202 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T00:59:09.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T00:59:09.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T00:59:09.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:09.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:09.207 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.550346844000003, 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T00:59:09.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.898058839000001, 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:09.212 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:09.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:09.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:09.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T00:59:09.217 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.218 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.218 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.218 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:09.218 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:09.218 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:09.219 INFO:teuthology.orchestra.run.smithi042.stdout:3746 trunc from 0x94824fd to 0x7ba5c3c 2024-09-18T00:59:09.220 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:09.213+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:09.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:09.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:09.238 INFO:teuthology.orchestra.run.smithi042.stdout:3747 write 0xcda91a6 thru 0xcdb7ce4 (0xeb3f bytes) 2024-09-18T00:59:09.243 INFO:teuthology.orchestra.run.smithi042.stdout:3748 write 0x609a311 thru 0x609bded (0x1add bytes) 2024-09-18T00:59:09.245 INFO:teuthology.orchestra.run.smithi042.stdout:3749 write 0xbebd770 thru 0xbec7c68 (0xa4f9 bytes) 2024-09-18T00:59:09.249 INFO:teuthology.orchestra.run.smithi042.stdout:3750 read 0xb462884 thru 0xb466724 (0x3ea1 bytes) 2024-09-18T00:59:09.257 INFO:teuthology.orchestra.run.smithi042.stdout:3751 trunc from 0xcdb7ce5 to 0x76045f9 2024-09-18T00:59:09.291 INFO:teuthology.orchestra.run.smithi042.stdout:3752 writesame 0xead8a65 thru 0xeae2564 data_size 0x28(0x9b00 bytes) 2024-09-18T00:59:09.291 INFO:teuthology.orchestra.run.smithi042.stdout:3753 punch from 0x7511114 to 0x751354a, (0x2436 bytes) 2024-09-18T00:59:09.305 INFO:teuthology.orchestra.run.smithi042.stdout:3754 trunc from 0xeae2565 to 0x526f2be 2024-09-18T00:59:09.320 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:09.317+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:09.332 INFO:teuthology.orchestra.run.smithi042.stdout:3755 punch from 0x425bfbc to 0x42612f8, (0x533c bytes) 2024-09-18T00:59:09.346 INFO:teuthology.orchestra.run.smithi042.stdout:3756 writesame 0x5bacdc5 thru 0x5bb6ea2 data_size 0x3b(0xa0de bytes) 2024-09-18T00:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout:3757 trunc from 0x5bb6ea3 to 0x5316382 2024-09-18T00:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:09.362 INFO:teuthology.orchestra.run.smithi042.stdout:3760 read 0x16ee582 thru 0x16f9aa8 (0xb527 bytes) 2024-09-18T00:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:09.374 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:09.382 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:09.382 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:09.393 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:09.393 INFO:teuthology.orchestra.run.smithi042.stdout:3761 write 0xec9464c thru 0xec95040 (0x9f5 bytes) 2024-09-18T00:59:09.398 INFO:teuthology.orchestra.run.smithi042.stdout:3762 writesame 0x73125c9 thru 0x731bdd5 data_size 0x4b(0x980d bytes) 2024-09-18T00:59:09.400 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:09.400 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:09.404 INFO:teuthology.orchestra.run.smithi042.stdout:3764 write 0xda32f1e thru 0xda3c49f (0x9582 bytes) 2024-09-18T00:59:09.409 INFO:teuthology.orchestra.run.smithi042.stdout:3765 write 0x3a89745 thru 0x3a971b5 (0xda71 bytes) 2024-09-18T00:59:09.411 INFO:teuthology.orchestra.run.smithi042.stdout:3769 write 0x6876469 thru 0x688611a (0xfcb2 bytes) 2024-09-18T00:59:09.417 INFO:teuthology.orchestra.run.smithi042.stdout:3770 punch from 0x387dd31 to 0x3884eb7, (0x7186 bytes) 2024-09-18T00:59:09.421 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:09.421 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:09.421 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:09.417+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:09.424 INFO:teuthology.orchestra.run.smithi042.stdout:3771 trunc from 0xec95041 to 0x9c8f4f0 2024-09-18T00:59:09.440 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:09.441 INFO:teuthology.orchestra.run.smithi042.stdout:3772 write 0x44b5cd5 thru 0x44c39a1 (0xdccd bytes) 2024-09-18T00:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout:3773 punch from 0x5fabf91 to 0x5fb6acd, (0xab3c bytes) 2024-09-18T00:59:09.444 INFO:teuthology.orchestra.run.smithi042.stdout:3775 write 0x992e5a0 thru 0x99372fa (0x8d5b bytes) 2024-09-18T00:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout:3776 read 0x5c79354 thru 0x5c86abf (0xd76c bytes) 2024-09-18T00:59:09.451 INFO:teuthology.orchestra.run.smithi042.stdout:3777 write 0x366aff5 thru 0x36799ff (0xea0b bytes) 2024-09-18T00:59:09.454 INFO:teuthology.orchestra.run.smithi042.stdout:3781 read 0x53d07c7 thru 0x53d1710 (0xf4a bytes) 2024-09-18T00:59:09.455 INFO:teuthology.orchestra.run.smithi042.stdout:3782 read 0x5f084e7 thru 0x5f0ac39 (0x2753 bytes) 2024-09-18T00:59:09.463 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:09.464 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:09.464 INFO:teuthology.orchestra.run.smithi042.stdout:3784 trunc from 0x9c8f4f0 to 0x36bba49 2024-09-18T00:59:09.486 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:09.486 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:09.492 INFO:teuthology.orchestra.run.smithi042.stdout:3785 punch from 0x1460d72 to 0x1469032, (0x82c0 bytes) 2024-09-18T00:59:09.508 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:09.508 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:09.512 INFO:teuthology.orchestra.run.smithi042.stdout:3786 writesame 0x832183b thru 0x8330baf data_size 0x4b(0xf375 bytes) 2024-09-18T00:59:09.512 INFO:teuthology.orchestra.run.smithi042.stdout:3788 punch from 0x318d57d to 0x3197f63, (0xa9e6 bytes) 2024-09-18T00:59:09.512 INFO:teuthology.orchestra.run.smithi042.stdout:3789 read 0x6233405 thru 0x624181e (0xe41a bytes) 2024-09-18T00:59:09.512 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:09.512 INFO:teuthology.orchestra.run.smithi042.stdout:3793 write 0x2a7efa9 thru 0x2a80780 (0x17d8 bytes) 2024-09-18T00:59:09.522 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:09.521+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:09.526 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:09.526 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:09.532 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:09.555 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout:3795 punch from 0x7d48229 to 0x7d488b7, (0x68e bytes) 2024-09-18T00:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout:3796 write 0x779af5 thru 0x784e4c (0xb358 bytes) 2024-09-18T00:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout:3798 read 0x7e8c46f thru 0x7e9a899 (0xe42b bytes) 2024-09-18T00:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout:3799 read 0x43caa10 thru 0x43d7fe9 (0xd5da bytes) 2024-09-18T00:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout:3800 punch from 0x810898b to 0x8117b6a, (0xf1df bytes) 2024-09-18T00:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout:3801 write 0x9b5af39 thru 0x9b699bd (0xea85 bytes) 2024-09-18T00:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout:3803 read 0x6c1cb94 thru 0x6c20a40 (0x3ead bytes) 2024-09-18T00:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout:3804 punch from 0x218d6ab to 0x218fa0e, (0x2363 bytes) 2024-09-18T00:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout:3807 read 0x2b1e5e0 thru 0x2b29f54 (0xb975 bytes) 2024-09-18T00:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout:3808 writesame 0x2fbc6ed thru 0x2fc33bc data_size 0x8(0x6cd0 bytes) 2024-09-18T00:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout:3810 write 0x7ecb9de thru 0x7ed1fb1 (0x65d4 bytes) 2024-09-18T00:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout:3812 write 0x3107ef0 thru 0x3108255 (0x366 bytes) 2024-09-18T00:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout:3816 write 0x814ef2d thru 0x815c1c4 (0xd298 bytes) 2024-09-18T00:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout:3817 writesame 0x11582b4 thru 0x115db64 data_size 0x13(0x58b1 bytes) 2024-09-18T00:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout:3819 write 0x7d9c9b4 thru 0x7dac8a2 (0xfeef bytes) 2024-09-18T00:59:09.624 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:09.617+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:09.624 INFO:teuthology.orchestra.run.smithi042.stdout:3820 write 0xe9ba89d thru 0xe9c9547 (0xecab bytes) 2024-09-18T00:59:09.631 INFO:teuthology.orchestra.run.smithi042.stdout:3822 write 0xb64a5b4 thru 0xb64e799 (0x41e6 bytes) 2024-09-18T00:59:09.636 INFO:teuthology.orchestra.run.smithi042.stdout:3823 punch from 0x1b64cb to 0x1c44b6, (0xdfeb bytes) 2024-09-18T00:59:09.648 INFO:teuthology.orchestra.run.smithi042.stdout:3824 trunc from 0xe9c9548 to 0x637b542 2024-09-18T00:59:09.681 INFO:teuthology.orchestra.run.smithi042.stdout:3826 read 0x36ae651 thru 0x36af055 (0xa05 bytes) 2024-09-18T00:59:09.682 INFO:teuthology.orchestra.run.smithi042.stdout:3828 punch from 0x56ed6e5 to 0x56fc1bf, (0xeada bytes) 2024-09-18T00:59:09.682 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:09.683 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:09.684 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:09.701 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:09.702 INFO:teuthology.orchestra.run.smithi042.stdout:3832 writesame 0x95279a0 thru 0x952e242 data_size 0x1(0x68a3 bytes) 2024-09-18T00:59:09.702 INFO:teuthology.orchestra.run.smithi042.stdout:3833 write 0x8b85eb4 thru 0x8b8ee6a (0x8fb7 bytes) 2024-09-18T00:59:09.708 INFO:teuthology.orchestra.run.smithi042.stdout:3834 read 0x8f3ec3 thru 0x8fef8c (0xb0ca bytes) 2024-09-18T00:59:09.709 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:09.713 INFO:teuthology.orchestra.run.smithi042.stdout:3838 trunc from 0x952e243 to 0x6cd6937 2024-09-18T00:59:09.724 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:09.717+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:09.728 INFO:teuthology.orchestra.run.smithi042.stdout:3839 punch from 0x2248c0d to 0x22543d1, (0xb7c4 bytes) 2024-09-18T00:59:09.728 INFO:teuthology.orchestra.run.smithi042.stdout:3843 punch from 0x46370ee to 0x464249b, (0xb3ad bytes) 2024-09-18T00:59:09.736 INFO:teuthology.orchestra.run.smithi042.stdout:3844 trunc from 0x6cd6937 to 0x22c29b5 2024-09-18T00:59:09.825 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:09.821+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:09.832 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:09.832 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:09.848 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:09.866 INFO:teuthology.orchestra.run.smithi042.stdout:3847 write 0x7d8edb6 thru 0x7d91006 (0x2251 bytes) 2024-09-18T00:59:09.878 INFO:teuthology.orchestra.run.smithi042.stdout:3849 trunc from 0x7d91007 to 0x5ab6bb8 2024-09-18T00:59:09.901 INFO:teuthology.orchestra.run.smithi042.stdout:3852 write 0x55e615a thru 0x55ecd00 (0x6ba7 bytes) 2024-09-18T00:59:09.916 INFO:teuthology.orchestra.run.smithi042.stdout:3853 trunc from 0x5ab6bb8 to 0x258c408 2024-09-18T00:59:09.926 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:09.921+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:09.943 INFO:teuthology.orchestra.run.smithi042.stdout:3855 read 0x18d5ba0 thru 0x18e23d9 (0xc83a bytes) 2024-09-18T00:59:09.977 INFO:teuthology.orchestra.run.smithi042.stdout:3857 writesame 0xa1c2503 thru 0xa1c56cd data_size 0x7(0x31cb bytes) 2024-09-18T00:59:09.994 INFO:teuthology.orchestra.run.smithi042.stdout:3858 trunc from 0xa1c56ce to 0x2984599 2024-09-18T00:59:10.002 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:10.002 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:10.009 INFO:teuthology.orchestra.run.smithi042.stdout:3859 read 0x115c671 thru 0x115f04f (0x29df bytes) 2024-09-18T00:59:10.022 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:10.024 INFO:teuthology.orchestra.run.smithi042.stdout:3861 trunc from 0x2984599 to 0x9f0dc2e 2024-09-18T00:59:10.026 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:10.021+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:10.035 INFO:teuthology.orchestra.run.smithi042.stdout:3863 read 0x9784c02 thru 0x978d28a (0x8689 bytes) 2024-09-18T00:59:10.038 INFO:teuthology.orchestra.run.smithi042.stdout:3864 trunc from 0x9f0dc2e to 0xb018016 2024-09-18T00:59:10.051 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.051 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.066 INFO:teuthology.orchestra.run.smithi042.stdout:3868 write 0xbf131e8 thru 0xbf20b32 (0xd94b bytes) 2024-09-18T00:59:10.074 INFO:teuthology.orchestra.run.smithi042.stdout:3869 writesame 0x7d0c0a7 thru 0x7d13d2a data_size 0x1a(0x7c84 bytes) 2024-09-18T00:59:10.082 INFO:teuthology.orchestra.run.smithi042.stdout:3871 write 0xb6cbabb thru 0xb6d9e30 (0xe376 bytes) 2024-09-18T00:59:10.089 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.089 INFO:teuthology.orchestra.run.smithi042.stdout:3874 write 0xddae7f thru 0xdde768 (0x38ea bytes) 2024-09-18T00:59:10.092 INFO:teuthology.orchestra.run.smithi042.stdout:3875 write 0x94658 thru 0x9c061 (0x7a0a bytes) 2024-09-18T00:59:10.096 INFO:teuthology.orchestra.run.smithi042.stdout:3879 writesame 0x8135410 thru 0x813fe2d data_size 0x43(0xaa1e bytes) 2024-09-18T00:59:10.122 INFO:teuthology.orchestra.run.smithi042.stdout:3881 trunc from 0xbf20b33 to 0x368b928 2024-09-18T00:59:10.127 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:10.125+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:10.147 INFO:teuthology.orchestra.run.smithi042.stdout:3882 write 0xe01d3b thru 0xe03e57 (0x211d bytes) 2024-09-18T00:59:10.150 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:10.150 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:10.151 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.151 INFO:teuthology.orchestra.run.smithi042.stdout:3884 write 0x6d6c6f thru 0x6e6501 (0xf893 bytes) 2024-09-18T00:59:10.155 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.155 INFO:teuthology.orchestra.run.smithi042.stdout:3886 writesame 0x2dc830f thru 0x2dc8c5a data_size 0x14(0x94c bytes) 2024-09-18T00:59:10.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:10.197 INFO:teuthology.orchestra.run.smithi042.stdout:3887 write 0xe100c2e thru 0xe10745a (0x682d bytes) 2024-09-18T00:59:10.216 INFO:teuthology.orchestra.run.smithi042.stdout:3888 write 0xe2f0a95 thru 0xe2f15b6 (0xb22 bytes) 2024-09-18T00:59:10.226 INFO:teuthology.orchestra.run.smithi042.stdout:3890 trunc from 0xe2f15b7 to 0xaa686c9 2024-09-18T00:59:10.228 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:10.225+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:10.247 INFO:teuthology.orchestra.run.smithi042.stdout:3891 punch from 0x2722298 to 0x27249d5, (0x273d bytes) 2024-09-18T00:59:10.247 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.247 INFO:teuthology.orchestra.run.smithi042.stdout:3893 punch from 0xa7d3690 to 0xa7e06ba, (0xd02a bytes) 2024-09-18T00:59:10.261 INFO:teuthology.orchestra.run.smithi042.stdout:3894 writesame 0xbb0bfd8 thru 0xbb12bbf data_size 0xc(0x6be8 bytes) 2024-09-18T00:59:10.261 INFO:teuthology.orchestra.run.smithi042.stdout:3895 punch from 0x96193ab to 0x961d38e, (0x3fe3 bytes) 2024-09-18T00:59:10.261 INFO:teuthology.orchestra.run.smithi042.stdout:3896 read 0x51449f2 thru 0x5152b64 (0xe173 bytes) 2024-09-18T00:59:10.262 INFO:teuthology.orchestra.run.smithi042.stdout:3897 write 0x3e5093d thru 0x3e513e3 (0xaa7 bytes) 2024-09-18T00:59:10.271 INFO:teuthology.orchestra.run.smithi042.stdout:3898 trunc from 0xbb12bc0 to 0xdef1fee 2024-09-18T00:59:10.285 INFO:teuthology.orchestra.run.smithi042.stdout:3899 write 0x3b5ae44 thru 0x3b66e3b (0xbff8 bytes) 2024-09-18T00:59:10.292 INFO:teuthology.orchestra.run.smithi042.stdout:3900 punch from 0xc51a8f8 to 0xc51c0c7, (0x17cf bytes) 2024-09-18T00:59:10.292 INFO:teuthology.orchestra.run.smithi042.stdout:3902 read 0x9b4b2cd thru 0x9b4c543 (0x1277 bytes) 2024-09-18T00:59:10.292 INFO:teuthology.orchestra.run.smithi042.stdout:3904 write 0x3874eb3 thru 0x387a335 (0x5483 bytes) 2024-09-18T00:59:10.316 INFO:teuthology.orchestra.run.smithi042.stdout:3905 trunc from 0xdef1fee to 0x1ac59a0 2024-09-18T00:59:10.330 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:10.329+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:10.354 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:10.354 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:10.364 INFO:teuthology.orchestra.run.smithi042.stdout:3906 trunc from 0x1ac59a0 to 0xbc91425 2024-09-18T00:59:10.378 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:10.397 INFO:teuthology.orchestra.run.smithi042.stdout:3907 write 0xd4c9353 thru 0xd4cb5d3 (0x2281 bytes) 2024-09-18T00:59:10.403 INFO:teuthology.orchestra.run.smithi042.stdout:3908 read 0x1f916b5 thru 0x1f98f9a (0x78e6 bytes) 2024-09-18T00:59:10.404 INFO:teuthology.orchestra.run.smithi042.stdout:3909 write 0xbb3098d thru 0xbb3c0d4 (0xb748 bytes) 2024-09-18T00:59:10.411 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.411 INFO:teuthology.orchestra.run.smithi042.stdout:3911 writesame 0x4f06a6c thru 0x4f118f3 data_size 0x14(0xae88 bytes) 2024-09-18T00:59:10.419 INFO:teuthology.orchestra.run.smithi042.stdout:3912 write 0x6e5ab04 thru 0x6e66c0c (0xc109 bytes) 2024-09-18T00:59:10.427 INFO:teuthology.orchestra.run.smithi042.stdout:3913 punch from 0x7e9e543 to 0x7eaa44e, (0xbf0b bytes) 2024-09-18T00:59:10.427 INFO:teuthology.orchestra.run.smithi042.stdout:3915 writesame 0x1cac8e6 thru 0x1cb9a7b data_size 0x8b(0xd196 bytes) 2024-09-18T00:59:10.429 INFO:teuthology.orchestra.run.smithi042.stdout:3916 write 0x8d700df thru 0x8d7f6bf (0xf5e1 bytes) 2024-09-18T00:59:10.431 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:10.425+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:10.435 INFO:teuthology.orchestra.run.smithi042.stdout:3920 read 0x6cdb876 thru 0x6ce7c9e (0xc429 bytes) 2024-09-18T00:59:10.436 INFO:teuthology.orchestra.run.smithi042.stdout:3921 read 0x4f9468c thru 0x4f9c72a (0x809f bytes) 2024-09-18T00:59:10.437 INFO:teuthology.orchestra.run.smithi042.stdout:3922 write 0x371a558 thru 0x3722be6 (0x868f bytes) 2024-09-18T00:59:10.440 INFO:teuthology.orchestra.run.smithi042.stdout:3924 write 0xa9b5dc9 thru 0xa9bb90f (0x5b47 bytes) 2024-09-18T00:59:10.447 INFO:teuthology.orchestra.run.smithi042.stdout:3925 read 0x10312a2 thru 0x103e77c (0xd4db bytes) 2024-09-18T00:59:10.448 INFO:teuthology.orchestra.run.smithi042.stdout:3926 write 0x84b13b1 thru 0x84b875c (0x73ac bytes) 2024-09-18T00:59:10.451 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.451 INFO:teuthology.orchestra.run.smithi042.stdout:3928 read 0x2b8fa49 thru 0x2b9dbe6 (0xe19e bytes) 2024-09-18T00:59:10.452 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.465 INFO:teuthology.orchestra.run.smithi042.stdout:3932 trunc from 0xd4cb5d4 to 0x4d5eec1 2024-09-18T00:59:10.527 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:10.527 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:10.528 INFO:teuthology.orchestra.run.smithi042.stdout:3933 write 0xe1ba5fd thru 0xe1c5899 (0xb29d bytes) 2024-09-18T00:59:10.531 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:10.525+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:10.535 INFO:teuthology.orchestra.run.smithi042.stdout:3936 read 0x45fa0b9 thru 0x45fe2c4 (0x420c bytes) 2024-09-18T00:59:10.535 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.536 INFO:teuthology.orchestra.run.smithi042.stdout:3938 write 0x93f522e thru 0x93fbc1c (0x69ef bytes) 2024-09-18T00:59:10.542 INFO:teuthology.orchestra.run.smithi042.stdout:3939 write 0xdae3f0d thru 0xdae5a5c (0x1b50 bytes) 2024-09-18T00:59:10.544 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:10.549 INFO:teuthology.orchestra.run.smithi042.stdout:3943 read 0xc70077e thru 0xc7088ac (0x812f bytes) 2024-09-18T00:59:10.549 INFO:teuthology.orchestra.run.smithi042.stdout:3945 writesame 0x1531cb3 thru 0x1535200 data_size 0x1(0x354e bytes) 2024-09-18T00:59:10.567 INFO:teuthology.orchestra.run.smithi042.stdout:3947 writesame 0x970c92e thru 0x97164a3 data_size 0x43(0x9b76 bytes) 2024-09-18T00:59:10.573 INFO:teuthology.orchestra.run.smithi042.stdout:3948 read 0x1eee170 thru 0x1ef8658 (0xa4e9 bytes) 2024-09-18T00:59:10.586 INFO:teuthology.orchestra.run.smithi042.stdout:3950 write 0xec7f73a thru 0xec81188 (0x1a4f bytes) 2024-09-18T00:59:10.589 INFO:teuthology.orchestra.run.smithi042.stdout:3953 write 0x8723c0a thru 0x872ec9e (0xb095 bytes) 2024-09-18T00:59:10.597 INFO:teuthology.orchestra.run.smithi042.stdout:3955 write 0xabc5cae thru 0xabcf326 (0x9679 bytes) 2024-09-18T00:59:10.600 INFO:teuthology.orchestra.run.smithi042.stdout:3956 read 0xcae533d thru 0xcaef8a3 (0xa567 bytes) 2024-09-18T00:59:10.601 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.601 INFO:teuthology.orchestra.run.smithi042.stdout:3959 read 0xded14a6 thru 0xded9691 (0x81ec bytes) 2024-09-18T00:59:10.601 INFO:teuthology.orchestra.run.smithi042.stdout:3960 read 0xcd4f3c6 thru 0xcd5cad4 (0xd70f bytes) 2024-09-18T00:59:10.602 INFO:teuthology.orchestra.run.smithi042.stdout:3961 write 0xdf8637 thru 0xe013ee (0x8db8 bytes) 2024-09-18T00:59:10.605 INFO:teuthology.orchestra.run.smithi042.stdout:3962 write 0x1bb83d5 thru 0x1bc1dc1 (0x99ed bytes) 2024-09-18T00:59:10.608 INFO:teuthology.orchestra.run.smithi042.stdout:3963 read 0xc1aca47 thru 0xc1b3e16 (0x73d0 bytes) 2024-09-18T00:59:10.609 INFO:teuthology.orchestra.run.smithi042.stdout:3964 trunc from 0xec81189 to 0xe806c23 2024-09-18T00:59:10.624 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.624 INFO:teuthology.orchestra.run.smithi042.stdout:3967 write 0xa3e45ae thru 0xa3e66a2 (0x20f5 bytes) 2024-09-18T00:59:10.627 INFO:teuthology.orchestra.run.smithi042.stdout:3969 writesame 0x4c69397 thru 0x4c749f2 data_size 0x16(0xb65c bytes) 2024-09-18T00:59:10.631 INFO:teuthology.orchestra.run.smithi042.stdout:3970 read 0x2c801b6 thru 0x2c89279 (0x90c4 bytes) 2024-09-18T00:59:10.632 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:10.625+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:10.632 INFO:teuthology.orchestra.run.smithi042.stdout:3972 punch from 0xc1e95fa to 0xc1ef0cb, (0x5ad1 bytes) 2024-09-18T00:59:10.633 INFO:teuthology.orchestra.run.smithi042.stdout:3973 punch from 0xceffdcf to 0xcf03b20, (0x3d51 bytes) 2024-09-18T00:59:10.633 INFO:teuthology.orchestra.run.smithi042.stdout:3974 punch from 0xb1bf34f to 0xb1ce864, (0xf515 bytes) 2024-09-18T00:59:10.633 INFO:teuthology.orchestra.run.smithi042.stdout:3975 write 0x6cda8fa thru 0x6ce5c76 (0xb37d bytes) 2024-09-18T00:59:10.636 INFO:teuthology.orchestra.run.smithi042.stdout:3978 write 0xa19840 thru 0xa286fd (0xeebe bytes) 2024-09-18T00:59:10.641 INFO:teuthology.orchestra.run.smithi042.stdout:3979 read 0xb626e69 thru 0xb63623b (0xf3d3 bytes) 2024-09-18T00:59:10.649 INFO:teuthology.orchestra.run.smithi042.stdout:3980 trunc from 0xe806c23 to 0x95e0965 2024-09-18T00:59:10.676 INFO:teuthology.orchestra.run.smithi042.stdout:3981 write 0x6d57a4f thru 0x6d60ca3 (0x9255 bytes) 2024-09-18T00:59:10.686 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:10.686 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:10.701 INFO:teuthology.orchestra.run.smithi042.stdout:3982 write 0xc02a3df thru 0xc033659 (0x927b bytes) 2024-09-18T00:59:10.702 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:10.708 INFO:teuthology.orchestra.run.smithi042.stdout:3983 write 0x17bcf9e thru 0x17c868c (0xb6ef bytes) 2024-09-18T00:59:10.727 INFO:teuthology.orchestra.run.smithi042.stdout:3984 writesame 0xd43a991 thru 0xd442f90 data_size 0x80(0x8600 bytes) 2024-09-18T00:59:10.727 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.727 INFO:teuthology.orchestra.run.smithi042.stdout:3988 punch from 0x9784037 to 0x9792d93, (0xed5c bytes) 2024-09-18T00:59:10.728 INFO:teuthology.orchestra.run.smithi042.stdout:3989 punch from 0xbff0864 to 0xbffa27f, (0x9a1b bytes) 2024-09-18T00:59:10.728 INFO:teuthology.orchestra.run.smithi042.stdout:3990 punch from 0x43abac8 to 0x43b5f2f, (0xa467 bytes) 2024-09-18T00:59:10.728 INFO:teuthology.orchestra.run.smithi042.stdout:3991 write 0x84d9195 thru 0x84e2173 (0x8fdf bytes) 2024-09-18T00:59:10.731 INFO:teuthology.orchestra.run.smithi042.stdout:3993 write 0xb14b70c thru 0xb154e78 (0x976d bytes) 2024-09-18T00:59:10.732 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:10.725+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:10.738 INFO:teuthology.orchestra.run.smithi042.stdout:3994 read 0xb91b72f thru 0xb928676 (0xcf48 bytes) 2024-09-18T00:59:10.739 INFO:teuthology.orchestra.run.smithi042.stdout:3995 write 0xc27fbe8 thru 0xc28aacf (0xaee8 bytes) 2024-09-18T00:59:10.741 INFO:teuthology.orchestra.run.smithi042.stdout:3996 read 0x1f5f3fa thru 0x1f6c767 (0xd36e bytes) 2024-09-18T00:59:10.742 INFO:teuthology.orchestra.run.smithi042.stdout:3997 read 0xc064b07 thru 0xc0665c1 (0x1abb bytes) 2024-09-18T00:59:10.742 INFO:teuthology.orchestra.run.smithi042.stdout:3998 read 0x330ff85 thru 0x3319f66 (0x9fe2 bytes) 2024-09-18T00:59:10.742 INFO:teuthology.orchestra.run.smithi042.stdout:4000 read 0xa4898b7 thru 0xa48e458 (0x4ba2 bytes) 2024-09-18T00:59:10.744 INFO:teuthology.orchestra.run.smithi042.stdout:4001 write 0x95c04b1 thru 0x95c4ccd (0x481d bytes) 2024-09-18T00:59:10.752 INFO:teuthology.orchestra.run.smithi042.stdout:4002 trunc from 0xd442f91 to 0xa4833e3 2024-09-18T00:59:10.793 INFO:teuthology.orchestra.run.smithi042.stdout:4004 writesame 0xcae324d thru 0xcaf0d9d data_size 0x5f(0xdb51 bytes) 2024-09-18T00:59:10.793 INFO:teuthology.orchestra.run.smithi042.stdout:4006 write 0x6383e2d thru 0x6386ddf (0x2fb3 bytes) 2024-09-18T00:59:10.797 INFO:teuthology.orchestra.run.smithi042.stdout:4007 write 0xa152fc7 thru 0xa1624d4 (0xf50e bytes) 2024-09-18T00:59:10.800 INFO:teuthology.orchestra.run.smithi042.stdout:4008 write 0x15ab2d9 thru 0x15b806c (0xcd94 bytes) 2024-09-18T00:59:10.803 INFO:teuthology.orchestra.run.smithi042.stdout:4011 write 0x42d149d thru 0x42d5dd4 (0x4938 bytes) 2024-09-18T00:59:10.806 INFO:teuthology.orchestra.run.smithi042.stdout:4012 punch from 0xa559cc9 to 0xa567f63, (0xe29a bytes) 2024-09-18T00:59:10.808 INFO:teuthology.orchestra.run.smithi042.stdout:4013 punch from 0xb0e1f9b to 0xb0eeff0, (0xd055 bytes) 2024-09-18T00:59:10.815 INFO:teuthology.orchestra.run.smithi042.stdout:4014 trunc from 0xcaf0d9e to 0x71580a9 2024-09-18T00:59:10.833 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:10.829+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:10.841 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:10.857 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:10.857 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:10.861 INFO:teuthology.orchestra.run.smithi042.stdout:4016 writesame 0xb2ad91c thru 0xb2af6c1 data_size 0x42(0x1da6 bytes) 2024-09-18T00:59:10.874 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:10.874 INFO:teuthology.orchestra.run.smithi042.stdout:4017 writesame 0xb74a064 thru 0xb756d94 data_size 0x1(0xcd31 bytes) 2024-09-18T00:59:10.875 INFO:teuthology.orchestra.run.smithi042.stdout:4018 read 0x926df9f thru 0x927a654 (0xc6b6 bytes) 2024-09-18T00:59:10.875 INFO:teuthology.orchestra.run.smithi042.stdout:4020 write 0x2910a1a thru 0x2915540 (0x4b27 bytes) 2024-09-18T00:59:10.888 INFO:teuthology.orchestra.run.smithi042.stdout:4022 trunc from 0xb756d95 to 0x3758aa1 2024-09-18T00:59:10.913 INFO:teuthology.orchestra.run.smithi042.stdout:4023 writesame 0xb56daa thru 0xb645f0 data_size 0x1(0xd847 bytes) 2024-09-18T00:59:10.933 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:10.929+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:10.986 INFO:teuthology.orchestra.run.smithi042.stdout:4026 clone 33 order 22 su 262144 sc 14 2024-09-18T00:59:10.997 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:10.997 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:11.011 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:11.035 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:11.029+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:11.037 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:59:11.131 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone32 from 0x3758aa1 (overlap 0x1ac59a0) to 0x2ad5c3 2024-09-18T00:59:11.136 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:11.129+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:11.187 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:11.187 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:11.205 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-18T00:59:11.253 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:11.249+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:11.338 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:11.337+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:11.340 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:11.340 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:11.355 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:11.439 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:11.437+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:11.520 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:11.520 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:11.535 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:11.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:11.559 INFO:teuthology.orchestra.run.smithi042.stdout:4028 write 0xba72546 thru 0xba7a7e8 (0x82a3 bytes) 2024-09-18T00:59:11.564 INFO:teuthology.orchestra.run.smithi042.stdout:4029 read 0x8ac04a7 thru 0x8acc27f (0xbdd9 bytes) 2024-09-18T00:59:11.582 INFO:teuthology.orchestra.run.smithi042.stdout:4030 trunc from 0xba7a7e9 to 0x28dcff 2024-09-18T00:59:11.584 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:11.581+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:11.639 INFO:teuthology.orchestra.run.smithi042.stdout:4031 writesame 0x286fe89 thru 0x2878e10 data_size 0xc(0x8f88 bytes) 2024-09-18T00:59:11.639 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:11.637+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:11.640 INFO:teuthology.orchestra.run.smithi042.stdout:4033 punch from 0x16082d5 to 0x16174de, (0xf209 bytes) 2024-09-18T00:59:11.640 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:11.640 INFO:teuthology.orchestra.run.smithi042.stdout:4037 punch from 0x1401e7 to 0x14a0dd, (0x9ef6 bytes) 2024-09-18T00:59:11.664 INFO:teuthology.orchestra.run.smithi042.stdout:4038 writesame 0xcf74d70 thru 0xcf7575f data_size 0xc(0x9f0 bytes) 2024-09-18T00:59:11.664 INFO:teuthology.orchestra.run.smithi042.stdout:4039 read 0xcba1aa8 thru 0xcba777c (0x5cd5 bytes) 2024-09-18T00:59:11.664 INFO:teuthology.orchestra.run.smithi042.stdout:4042 write 0x946e30 thru 0x949a57 (0x2c28 bytes) 2024-09-18T00:59:11.666 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:11.666 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:11.667 INFO:teuthology.orchestra.run.smithi042.stdout:4045 write 0x118e1a thru 0x124c92 (0xbe79 bytes) 2024-09-18T00:59:11.668 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:11.668 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:11.669 INFO:teuthology.orchestra.run.smithi042.stdout:4046 writesame 0xb096c21 thru 0xb09c645 data_size 0x1(0x5a25 bytes) 2024-09-18T00:59:11.683 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:11.694 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:11.694 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:11.694 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:11.694 INFO:teuthology.orchestra.run.smithi042.stdout:4051 write 0x9fdb7df thru 0x9fdbce2 (0x504 bytes) 2024-09-18T00:59:11.699 INFO:teuthology.orchestra.run.smithi042.stdout:4052 read 0xc516ff8 thru 0xc5177f3 (0x7fc bytes) 2024-09-18T00:59:11.721 INFO:teuthology.orchestra.run.smithi042.stdout:4053 trunc from 0xcf75760 to 0x18375bc 2024-09-18T00:59:11.735 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:11.735 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:11.737 INFO:teuthology.orchestra.run.smithi042.stdout:4054 read 0x1541446 thru 0x154b4ce (0xa089 bytes) 2024-09-18T00:59:11.740 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:11.737+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:11.753 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:11.753 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:11.762 INFO:teuthology.orchestra.run.smithi042.stdout:4055 write 0xd8cca26 thru 0xd8d67f5 (0x9dd0 bytes) 2024-09-18T00:59:11.766 INFO:teuthology.orchestra.run.smithi042.stdout:4059 read 0xaa3ea74 thru 0xaa4c199 (0xd726 bytes) 2024-09-18T00:59:11.766 INFO:teuthology.orchestra.run.smithi042.stdout:4060 write 0x1915676 thru 0x1915d7c (0x707 bytes) 2024-09-18T00:59:11.768 INFO:teuthology.orchestra.run.smithi042.stdout:4062 read 0xcd53df1 thru 0xcd59d6a (0x5f7a bytes) 2024-09-18T00:59:11.768 INFO:teuthology.orchestra.run.smithi042.stdout:4063 read 0x9f61983 thru 0x9f6909c (0x771a bytes) 2024-09-18T00:59:11.769 INFO:teuthology.orchestra.run.smithi042.stdout:4064 writesame 0x86ece47 thru 0x86f4d10 data_size 0x1(0x7eca bytes) 2024-09-18T00:59:11.771 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:11.788 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:11.791 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:11.792 INFO:teuthology.orchestra.run.smithi042.stdout:4067 read 0xaec93d9 thru 0xaecea3b (0x5663 bytes) 2024-09-18T00:59:11.792 INFO:teuthology.orchestra.run.smithi042.stdout:4068 read 0x40bab0f thru 0x40c098d (0x5e7f bytes) 2024-09-18T00:59:11.792 INFO:teuthology.orchestra.run.smithi042.stdout:4069 punch from 0x31d55a5 to 0x31e0477, (0xaed2 bytes) 2024-09-18T00:59:11.792 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:11.793 INFO:teuthology.orchestra.run.smithi042.stdout:4072 read 0x4a61658 thru 0x4a699cc (0x8375 bytes) 2024-09-18T00:59:11.793 INFO:teuthology.orchestra.run.smithi042.stdout:4073 trunc from 0xd8d67f6 to 0xdf52983 2024-09-18T00:59:11.800 INFO:teuthology.orchestra.run.smithi042.stdout:4075 write 0x31660c8 thru 0x316ac61 (0x4b9a bytes) 2024-09-18T00:59:11.802 INFO:teuthology.orchestra.run.smithi042.stdout:4076 writesame 0x47273cf thru 0x4731164 data_size 0x17(0x9d96 bytes) 2024-09-18T00:59:11.806 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:11.806 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:11.807 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:11.819 INFO:teuthology.orchestra.run.smithi042.stdout:4080 trunc from 0xdf52983 to 0x674df77 2024-09-18T00:59:11.825 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:11.825 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:11.834 INFO:teuthology.orchestra.run.smithi042.stdout:4081 read 0x3375f00 thru 0x337ff29 (0xa02a bytes) 2024-09-18T00:59:11.841 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:11.837+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:11.847 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:11.850 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:11.850 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:11.850 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:11.850 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:11.854 INFO:teuthology.orchestra.run.smithi042.stdout:4082 write 0xe149eaf thru 0xe14e490 (0x45e2 bytes) 2024-09-18T00:59:11.858 INFO:teuthology.orchestra.run.smithi042.stdout:4083 read 0x1041eaa thru 0x104bf37 (0xa08e bytes) 2024-09-18T00:59:11.859 INFO:teuthology.orchestra.run.smithi042.stdout:4085 punch from 0x2fcf350 to 0x2fdd142, (0xddf2 bytes) 2024-09-18T00:59:11.859 INFO:teuthology.orchestra.run.smithi042.stdout:4087 write 0x5a6debc thru 0x5a77ad5 (0x9c1a bytes) 2024-09-18T00:59:11.863 INFO:teuthology.orchestra.run.smithi042.stdout:4088 write 0x2f739fb thru 0x2f80a67 (0xd06d bytes) 2024-09-18T00:59:11.866 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:11.866 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:11.866 INFO:teuthology.orchestra.run.smithi042.stdout:4091 punch from 0x8d2d68e to 0x8d33fa3, (0x6915 bytes) 2024-09-18T00:59:11.867 INFO:teuthology.orchestra.run.smithi042.stdout:4092 punch from 0x4d0ca87 to 0x4d0da88, (0x1001 bytes) 2024-09-18T00:59:11.867 INFO:teuthology.orchestra.run.smithi042.stdout:4093 read 0x52c8e86 thru 0x52d229a (0x9415 bytes) 2024-09-18T00:59:11.867 INFO:teuthology.orchestra.run.smithi042.stdout:4094 read 0x13174e4 thru 0x1327285 (0xfda2 bytes) 2024-09-18T00:59:11.867 INFO:teuthology.orchestra.run.smithi042.stdout:4095 write 0xde6236 thru 0xdf1c3b (0xba06 bytes) 2024-09-18T00:59:11.868 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:11.871 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:11.871 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:11.871 INFO:teuthology.orchestra.run.smithi042.stdout:4096 read 0x2526bae thru 0x2532817 (0xbc6a bytes) 2024-09-18T00:59:11.872 INFO:teuthology.orchestra.run.smithi042.stdout:4097 read 0x6515ad6 thru 0x6516fef (0x151a bytes) 2024-09-18T00:59:11.872 INFO:teuthology.orchestra.run.smithi042.stdout:4098 read 0x22e53c4 thru 0x22ea380 (0x4fbd bytes) 2024-09-18T00:59:11.873 INFO:teuthology.orchestra.run.smithi042.stdout:4099 write 0xb5cbd53 thru 0xb5da436 (0xe6e4 bytes) 2024-09-18T00:59:11.881 INFO:teuthology.orchestra.run.smithi042.stdout:4101 trunc from 0xe14e491 to 0xbc1f2be 2024-09-18T00:59:11.895 INFO:teuthology.orchestra.run.smithi042.stdout:4103 write 0x623a604 thru 0x623b055 (0xa52 bytes) 2024-09-18T00:59:11.899 INFO:teuthology.orchestra.run.smithi042.stdout:4104 write 0x5b99793 thru 0x5b9f321 (0x5b8f bytes) 2024-09-18T00:59:11.903 INFO:teuthology.orchestra.run.smithi042.stdout:4105 read 0xfc44ca thru 0xfc7a96 (0x35cd bytes) 2024-09-18T00:59:11.903 INFO:teuthology.orchestra.run.smithi042.stdout:4106 read 0x83be501 thru 0x83c94f2 (0xaff2 bytes) 2024-09-18T00:59:11.903 INFO:teuthology.orchestra.run.smithi042.stdout:4107 read 0x3eae6e2 thru 0x3eb594a (0x7269 bytes) 2024-09-18T00:59:11.904 INFO:teuthology.orchestra.run.smithi042.stdout:4110 read 0xb6f5ab8 thru 0xb6ff2f0 (0x9839 bytes) 2024-09-18T00:59:11.904 INFO:teuthology.orchestra.run.smithi042.stdout:4111 punch from 0x9648ea9 to 0x96559d2, (0xcb29 bytes) 2024-09-18T00:59:11.904 INFO:teuthology.orchestra.run.smithi042.stdout:4113 writesame 0x3ea9618 thru 0x3eb6454 data_size 0x1(0xce3d bytes) 2024-09-18T00:59:11.937 INFO:teuthology.orchestra.run.smithi042.stdout:4114 read 0x423fb4d thru 0x42424cb (0x297f bytes) 2024-09-18T00:59:11.942 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:11.941+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:11.947 INFO:teuthology.orchestra.run.smithi042.stdout:4115 writesame 0xc601b2d thru 0xc60cfe8 data_size 0x56(0xb4bc bytes) 2024-09-18T00:59:11.947 INFO:teuthology.orchestra.run.smithi042.stdout:4119 write 0x1e90e8e thru 0x1e9e624 (0xd797 bytes) 2024-09-18T00:59:11.968 INFO:teuthology.orchestra.run.smithi042.stdout:4120 trunc from 0xc60cfe9 to 0x10599b1 2024-09-18T00:59:11.998 INFO:teuthology.orchestra.run.smithi042.stdout:4121 trunc from 0x10599b1 to 0x3076b0 2024-09-18T00:59:12.001 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:12.001 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:12.001 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:12.001 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:12.001 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:12.019 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:12.032 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:59:12.042 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:12.041+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:4123 write 0x37b17f1 thru 0x37b5c94 (0x44a4 bytes) 2024-09-18T00:59:12.050 INFO:teuthology.orchestra.run.smithi042.stdout:4124 punch from 0x233a2e6 to 0x233e071, (0x3d8b bytes) 2024-09-18T00:59:12.050 INFO:teuthology.orchestra.run.smithi042.stdout:4125 read 0x2fd6c7b thru 0x2fe0323 (0x96a9 bytes) 2024-09-18T00:59:12.050 INFO:teuthology.orchestra.run.smithi042.stdout:4126 read 0xcc96d6 thru 0xcca3aa (0xcd5 bytes) 2024-09-18T00:59:12.051 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:12.065 INFO:teuthology.orchestra.run.smithi042.stdout:4128 trunc from 0x37b5c95 to 0xcd53a25 2024-09-18T00:59:12.071 INFO:teuthology.orchestra.run.smithi042.stdout:4129 read 0x5db9520 thru 0x5dc2206 (0x8ce7 bytes) 2024-09-18T00:59:12.072 INFO:teuthology.orchestra.run.smithi042.stdout:4130 read 0x8260a80 thru 0x8265e0b (0x538c bytes) 2024-09-18T00:59:12.088 INFO:teuthology.orchestra.run.smithi042.stdout:4132 trunc from 0xcd53a25 to 0x1a5e3af 2024-09-18T00:59:12.101 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:12.101 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:12.126 INFO:teuthology.orchestra.run.smithi042.stdout:4136 writesame 0xce00b22 thru 0xce0b1ca data_size 0x73(0xa6a9 bytes) 2024-09-18T00:59:12.126 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:12.126 INFO:teuthology.orchestra.run.smithi042.stdout:4139 read 0x66f9010 thru 0x6708a1e (0xfa0f bytes) 2024-09-18T00:59:12.135 INFO:teuthology.orchestra.run.smithi042.stdout:4140 trunc from 0xce0b1cb to 0x7244364 2024-09-18T00:59:12.140 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.730929283999998, 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:12.141 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T00:59:12.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T00:59:12.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T00:59:12.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:12.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T00:59:12.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T00:59:12.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:12.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T00:59:12.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125622 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 344064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673346+0000", 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.4667941170000001, 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096365904000000002, 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.147 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125622 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673347+0000", 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7599999999999997e-07 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673354+0000", 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.486e-06 2024-09-18T00:59:12.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673361+0000", 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5520000000000001e-06 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673365+0000", 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7129999999999996e-06 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768923+0000", 2024-09-18T00:59:12.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095557673999999995 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768994+0000", 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0847999999999997e-05 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769712+0000", 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000718055 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.151 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125623 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673371+0000", 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.4667697029999998, 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096098244999999999, 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125623 2024-09-18T00:59:12.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8200000000000001e-07 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673372+0000", 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5519999999999999e-06 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673376+0000", 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.512e-06 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673379+0000", 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1580000000000001e-06 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769200+0000", 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095820946000000004 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769317+0000", 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117003 2024-09-18T00:59:12.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769469+0000", 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015179200000000001 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125624 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673382+0000", 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.466757839, 2024-09-18T00:59:12.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101719451, 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125624 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T00:59:12.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673383+0000", 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.23e-07 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673384+0000", 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3060000000000001e-06 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673387+0000", 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2650000000000001e-06 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673390+0000", 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7810000000000001e-06 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769495+0000", 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096104599999999998 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769571+0000", 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6089999999999998e-05 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775102+0000", 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055311859999999996 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.160 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125625 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673393+0000", 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.4667467950000002, 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102242338, 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.161 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125625 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673394+0000", 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3700000000000001e-07 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673395+0000", 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2839999999999999e-06 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673398+0000", 2024-09-18T00:59:12.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1379999999999999e-06 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673401+0000", 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7149999999999998e-06 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775140+0000", 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101739127 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775212+0000", 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1984999999999996e-05 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775636+0000", 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000423752 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125627 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.165 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673404+0000", 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.4667358780000002, 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10248196399999999, 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125627 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673405+0000", 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0699999999999998e-07 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673406+0000", 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2130000000000001e-06 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673409+0000", 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3400000000000002e-06 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673412+0000", 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7360000000000001e-06 2024-09-18T00:59:12.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775665+0000", 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102253552 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775734+0000", 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8019e-05 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775886+0000", 2024-09-18T00:59:12.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015279700000000001 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125628 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673415+0000", 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.4667247799999998, 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.1031562, 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.170 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125628 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673416+0000", 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.91e-07 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673418+0000", 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9259999999999999e-06 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673423+0000", 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6149999999999996e-06 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673428+0000", 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1460000000000004e-06 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776149+0000", 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10272025899999999 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776218+0000", 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9207000000000003e-05 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776572+0000", 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035375599999999999 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125629 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673435+0000", 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.466705229, 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103390994, 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125629 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.175 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T00:59:12.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1800000000000001e-07 2024-09-18T00:59:12.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673438+0000", 2024-09-18T00:59:12.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4310000000000001e-06 2024-09-18T00:59:12.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673446+0000", 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0290000000000005e-06 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673451+0000", 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3630000000000002e-06 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776600+0000", 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10314894500000001 2024-09-18T00:59:12.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776668+0000", 2024-09-18T00:59:12.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7963e-05 2024-09-18T00:59:12.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776826+0000", 2024-09-18T00:59:12.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015784499999999999 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125631 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673458+0000", 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.4666819790000001, 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.104002186, 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.180 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125631 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T00:59:12.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673459+0000", 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4499999999999997e-07 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673461+0000", 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.351e-06 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673467+0000", 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3389999999999998e-06 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673472+0000", 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9490000000000004e-06 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777062+0000", 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103589771 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777130+0000", 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8065999999999995e-05 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777460+0000", 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000330265 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.184 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125632 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673479+0000", 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.466661191, 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10711156300000001, 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125632 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6100000000000002e-07 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673482+0000", 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1210000000000001e-06 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673488+0000", 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1659999999999999e-06 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673493+0000", 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.989e-06 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777489+0000", 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10399652600000001 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777565+0000", 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6219999999999999e-05 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780591+0000", 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030251800000000001 2024-09-18T00:59:12.188 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125633 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673500+0000", 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.4666404289999999, 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10735947699999999, 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.189 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.190 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.190 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125633 2024-09-18T00:59:12.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.190 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T00:59:12.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T00:59:12.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9099999999999999e-07 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673503+0000", 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4360000000000001e-06 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673509+0000", 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3969999999999999e-06 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673515+0000", 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7640000000000002e-06 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780622+0000", 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.107107533 2024-09-18T00:59:12.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780692+0000", 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9735999999999998e-05 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780859+0000", 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016721999999999999 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126363 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.711354+0000", 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.428786288, 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10567230599999999, 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126363 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711355+0000", 2024-09-18T00:59:12.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711360+0000", 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9740000000000001e-06 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711380+0000", 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9769e-05 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711396+0000", 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5947000000000002e-05 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817011+0000", 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.105615185 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817026+0000", 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5430999999999998e-05 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126375 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721909+0000", 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.418231638, 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095654577000000005, 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126375 2024-09-18T00:59:12.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721909+0000", 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721909+0000", 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721909+0000", 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3300000000000003e-07 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721912+0000", 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.683e-06 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721918+0000", 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2380000000000002e-06 2024-09-18T00:59:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721923+0000", 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9180000000000002e-06 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817557+0000", 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095633911000000002 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817563+0000", 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3940000000000001e-06 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126376 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721930+0000", 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.418210003, 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097651948000000002, 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126376 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T00:59:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721931+0000", 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721933+0000", 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721939+0000", 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721944+0000", 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0089999999999998e-06 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817590+0000", 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095646089000000004 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.819582+0000", 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019924669999999999 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126427 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721951+0000", 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.4181894700000002, 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095671891999999994, 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126427 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9499999999999998e-07 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721954+0000", 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4700000000000001e-06 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721960+0000", 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4629999999999998e-06 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721965+0000", 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0590000000000002e-06 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817614+0000", 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095649026999999998 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817623+0000", 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4780000000000008e-06 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126428 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721973+0000", 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.4181672829999998, 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095683878999999999, 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126428 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721976+0000", 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4279999999999999e-06 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721983+0000", 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7499999999999997e-06 2024-09-18T00:59:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721988+0000", 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4349999999999996e-06 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817644+0000", 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095656525000000006 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817657+0000", 2024-09-18T00:59:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2312e-05 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126429 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721996+0000", 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.418144243, 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095687929000000005, 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126429 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9400000000000001e-07 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721999+0000", 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.649e-06 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722006+0000", 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8299999999999998e-06 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722011+0000", 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6970000000000001e-06 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817678+0000", 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095667143999999996 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817684+0000", 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2149999999999998e-06 2024-09-18T00:59:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126430 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722018+0000", 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.41812212, 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095682481999999999, 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126430 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722019+0000", 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722021+0000", 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.419e-06 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722027+0000", 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0750000000000003e-06 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722033+0000", 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6030000000000004e-06 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817695+0000", 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095662607999999996 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817701+0000", 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1359999999999996e-06 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126431 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722040+0000", 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.4181002170000001, 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678416000000002, 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126431 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8000000000000001e-07 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722043+0000", 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722049+0000", 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3219999999999998e-06 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722054+0000", 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817715+0000", 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095660429000000005 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817718+0000", 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7349999999999998e-06 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126432 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722061+0000", 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.4180787960000001, 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095671141000000001, 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126432 2024-09-18T00:59:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722061+0000", 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722061+0000", 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722062+0000", 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5400000000000002e-07 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722064+0000", 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2730000000000001e-06 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722070+0000", 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.117e-06 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722075+0000", 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3310000000000003e-06 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817729+0000", 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095653338000000004 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817733+0000", 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7280000000000001e-06 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.232 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.232 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.232 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:12.234 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:12.234 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:12.229+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:12.235 INFO:teuthology.orchestra.run.smithi042.stdout:4141 trunc from 0x7244364 to 0x596ba6d 2024-09-18T00:59:12.235 INFO:teuthology.orchestra.run.smithi042.stdout:4142 read 0x13bf1ca thru 0x13cb50b (0xc342 bytes) 2024-09-18T00:59:12.235 INFO:teuthology.orchestra.run.smithi042.stdout:4143 read 0x1de630 thru 0x1e2713 (0x40e4 bytes) 2024-09-18T00:59:12.235 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:12.235 INFO:teuthology.orchestra.run.smithi042.stdout:4145 read 0x265804d thru 0x2664abe (0xca72 bytes) 2024-09-18T00:59:12.235 INFO:teuthology.orchestra.run.smithi042.stdout:4147 write 0x14d7f63 thru 0x14d82f6 (0x394 bytes) 2024-09-18T00:59:12.235 INFO:teuthology.orchestra.run.smithi042.stdout:4148 read 0x20d98b thru 0x215185 (0x77fb bytes) 2024-09-18T00:59:12.235 INFO:teuthology.orchestra.run.smithi042.stdout:4149 punch from 0x5638334 to 0x56413b7, (0x9083 bytes) 2024-09-18T00:59:12.235 INFO:teuthology.orchestra.run.smithi042.stdout:4151 write 0x896910e thru 0x896e41e (0x5311 bytes) 2024-09-18T00:59:12.235 INFO:teuthology.orchestra.run.smithi042.stdout:4153 read 0x1912091 thru 0x191e960 (0xc8d0 bytes) 2024-09-18T00:59:12.235 INFO:teuthology.orchestra.run.smithi042.stdout:4154 trunc from 0x896e41f to 0x19e0ae8 2024-09-18T00:59:12.236 INFO:teuthology.orchestra.run.smithi042.stdout:4155 punch from 0x143ab69 to 0x14427cd, (0x7c64 bytes) 2024-09-18T00:59:12.236 INFO:teuthology.orchestra.run.smithi042.stdout:4157 read 0x8e61f5 thru 0x8edca4 (0x7ab0 bytes) 2024-09-18T00:59:12.237 INFO:teuthology.orchestra.run.smithi042.stdout:4158 read 0x12a60a3 thru 0x12aa7c0 (0x471e bytes) 2024-09-18T00:59:12.269 INFO:teuthology.orchestra.run.smithi042.stdout:4159 writesame 0x7e6d29d thru 0x7e73347 data_size 0x3(0x60ab bytes) 2024-09-18T00:59:12.292 INFO:teuthology.orchestra.run.smithi042.stdout:4160 writesame 0xc5848af thru 0xc586407 data_size 0x1(0x1b59 bytes) 2024-09-18T00:59:12.313 INFO:teuthology.orchestra.run.smithi042.stdout:4161 writesame 0xebfd166 thru 0xec05e2f data_size 0x3(0x8cca bytes) 2024-09-18T00:59:12.313 INFO:teuthology.orchestra.run.smithi042.stdout:4162 read 0xbf76bf3 thru 0xbf790d3 (0x24e1 bytes) 2024-09-18T00:59:12.313 INFO:teuthology.orchestra.run.smithi042.stdout:4164 write 0x224aade thru 0x224dd7c (0x329f bytes) 2024-09-18T00:59:12.319 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:12.320 INFO:teuthology.orchestra.run.smithi042.stdout:4166 trunc from 0xec05e30 to 0xe318581 2024-09-18T00:59:12.334 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:12.333+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:12.335 INFO:teuthology.orchestra.run.smithi042.stdout:4167 writesame 0x4ac8225 thru 0x4acb235 data_size 0x17(0x3011 bytes) 2024-09-18T00:59:12.342 INFO:teuthology.orchestra.run.smithi042.stdout:4169 read 0x4a01c20 thru 0x4a0634e (0x472f bytes) 2024-09-18T00:59:12.342 INFO:teuthology.orchestra.run.smithi042.stdout:4171 writesame 0xcda086 thru 0xce0432 data_size 0x13(0x63ad bytes) 2024-09-18T00:59:12.346 INFO:teuthology.orchestra.run.smithi042.stdout:4173 write 0x3ff6a5f thru 0x3ff940b (0x29ad bytes) 2024-09-18T00:59:12.353 INFO:teuthology.orchestra.run.smithi042.stdout:4174 writesame 0x5f5ff17 thru 0x5f68cd8 data_size 0x26(0x8dc2 bytes) 2024-09-18T00:59:12.356 INFO:teuthology.orchestra.run.smithi042.stdout:4175 punch from 0x48dbeed to 0x48e94a7, (0xd5ba bytes) 2024-09-18T00:59:12.356 INFO:teuthology.orchestra.run.smithi042.stdout:4176 writesame 0xc2d1a5f thru 0xc2d81bb data_size 0x1(0x675d bytes) 2024-09-18T00:59:12.389 INFO:teuthology.orchestra.run.smithi042.stdout:4177 read 0x962d0f5 thru 0x9635fff (0x8f0b bytes) 2024-09-18T00:59:12.389 INFO:teuthology.orchestra.run.smithi042.stdout:4178 read 0xd7b6cc4 thru 0xd7c529a (0xe5d7 bytes) 2024-09-18T00:59:12.390 INFO:teuthology.orchestra.run.smithi042.stdout:4179 punch from 0x214d1c4 to 0x2158d9e, (0xbbda bytes) 2024-09-18T00:59:12.390 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:12.390 INFO:teuthology.orchestra.run.smithi042.stdout:4183 read 0x65a94a4 thru 0x65a9672 (0x1cf bytes) 2024-09-18T00:59:12.410 INFO:teuthology.orchestra.run.smithi042.stdout:4184 trunc from 0xe318581 to 0x6da14ac 2024-09-18T00:59:12.410 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:12.410 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:12.425 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:12.434 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:12.433+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:12.435 INFO:teuthology.orchestra.run.smithi042.stdout:4185 writesame 0x3d827ef thru 0x3d87b13 data_size 0x37(0x5325 bytes) 2024-09-18T00:59:12.444 INFO:teuthology.orchestra.run.smithi042.stdout:4186 writesame 0x5de720d thru 0x5df0257 data_size 0x7(0x904b bytes) 2024-09-18T00:59:12.454 INFO:teuthology.orchestra.run.smithi042.stdout:4188 write 0x5102cf4 thru 0x510c898 (0x9ba5 bytes) 2024-09-18T00:59:12.460 INFO:teuthology.orchestra.run.smithi042.stdout:4189 writesame 0x614ece8 thru 0x615a821 data_size 0x5(0xbb3a bytes) 2024-09-18T00:59:12.476 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:12.476 INFO:teuthology.orchestra.run.smithi042.stdout:4192 punch from 0x5293fd6 to 0x529bf66, (0x7f90 bytes) 2024-09-18T00:59:12.476 INFO:teuthology.orchestra.run.smithi042.stdout:4194 read 0x35cddd6 thru 0x35d8b86 (0xadb1 bytes) 2024-09-18T00:59:12.476 INFO:teuthology.orchestra.run.smithi042.stdout:4195 read 0xe97da7 thru 0xe9d9e8 (0x5c42 bytes) 2024-09-18T00:59:12.477 INFO:teuthology.orchestra.run.smithi042.stdout:4197 writesame 0x597b25d thru 0x597dd05 data_size 0x1(0x2aa9 bytes) 2024-09-18T00:59:12.489 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:12.513 INFO:teuthology.orchestra.run.smithi042.stdout:4200 write 0xaa23f89 thru 0xaa24d63 (0xddb bytes) 2024-09-18T00:59:12.519 INFO:teuthology.orchestra.run.smithi042.stdout:4202 clone 34 order 25 su 1048576 sc 11 2024-09-18T00:59:12.535 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:12.529+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:12.553 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:12.553 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:12.553 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:12.553 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:12.554 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:12.567 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:12.653 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:12.649+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:12.734 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:12.734 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:12.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:12.734 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:12.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.734 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T00:59:12.734 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:12.734 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 83.644595437999996, 2024-09-18T00:59:12.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:12.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T00:59:12.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T00:59:12.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 83.253700487000003, 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:12.739 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T00:59:12.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T00:59:12.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78518 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1183744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.902169657999998, 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054532499999999998, 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78518 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831040+0000", 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3099999999999997e-07 2024-09-18T00:59:12.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831042+0000", 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2879999999999999e-06 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831050+0000", 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.464e-06 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831055+0000", 2024-09-18T00:59:12.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0819999999999998e-06 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885354+0000", 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298739999999998 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885428+0000", 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4567999999999995e-05 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885572+0000", 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014372699999999999 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78519 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1204224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.902147808999999, 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054707261, 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.748 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78519 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831062+0000", 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831064+0000", 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0360000000000001e-06 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831069+0000", 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4149999999999998e-06 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831075+0000", 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2249999999999997e-06 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885602+0000", 2024-09-18T00:59:12.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054526893 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885673+0000", 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0865000000000007e-05 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885768+0000", 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5422999999999998e-05 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78520 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1224704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.902127353000001, 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054906695999999998, 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78520 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.753 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:12.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831084+0000", 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2960000000000001e-06 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831091+0000", 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831095+0000", 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.792e-06 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885802+0000", 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054706795000000003 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885884+0000", 2024-09-18T00:59:12.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2297000000000002e-05 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885988+0000", 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010393199999999999 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78521 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:12.757 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.902082480000001, 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055066616999999998, 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78521 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831127+0000", 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.99e-07 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831130+0000", 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6819999999999999e-06 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831137+0000", 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3849999999999996e-06 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831145+0000", 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7360000000000001e-06 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886020+0000", 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054874700999999998 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886096+0000", 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6234000000000002e-05 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886193+0000", 2024-09-18T00:59:12.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7280000000000004e-05 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78522 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.902056719000001, 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055137938999999997, 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.762 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78522 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831153+0000", 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.37e-07 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831155+0000", 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4420000000000001e-06 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831161+0000", 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831166+0000", 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8749999999999999e-06 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886279+0000", 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055112337999999997 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886290+0000", 2024-09-18T00:59:12.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1527e-05 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78523 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.902036038999999, 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055171425000000003, 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.766 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78523 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:12.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0900000000000002e-07 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831176+0000", 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4830000000000002e-06 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831183+0000", 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0679999999999999e-06 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831189+0000", 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886331+0000", 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055142669999999998 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886344+0000", 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2918999999999999e-05 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78524 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.902013922999998, 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055194430000000003, 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78524 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831196+0000", 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1200000000000003e-07 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831198+0000", 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831204+0000", 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2740000000000004e-06 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831209+0000", 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0900000000000004e-06 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886381+0000", 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055172142 2024-09-18T00:59:12.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886389+0000", 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2150000000000001e-06 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78525 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.901993409999999, 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055206535000000001, 2024-09-18T00:59:12.774 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78525 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832358+0000", 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001142531 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832364+0000", 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:59:12.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832375+0000", 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1381e-05 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832382+0000", 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9800000000000001e-06 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886415+0000", 2024-09-18T00:59:12.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032678000000001 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886422+0000", 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4449999999999999e-06 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108130 3.1 3:83363b4a:::rbd_data.117c8e0d79e4.000000000000002d:20 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.5392783399999992, 2024-09-18T00:59:12.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054351575999999999, 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108130 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1100000000000001e-07 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193933+0000", 2024-09-18T00:59:12.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3690000000000001e-06 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193940+0000", 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1460000000000001e-06 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193944+0000", 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9230000000000001e-06 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.245560+0000", 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051615019999999998 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.245633+0000", 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3490000000000003e-05 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.248282+0000", 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026492170000000002 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.782 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108393 3.1 3:86b5547c:::rbd_data.117c8e0d79e4.0000000000000037:20 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.5361867849999999, 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054339392, 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.783 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108393 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5499999999999999e-07 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197024+0000", 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.308e-06 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197029+0000", 2024-09-18T00:59:12.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197032+0000", 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.994e-06 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251285+0000", 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054253006999999999 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251322+0000", 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7104000000000003e-05 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251362+0000", 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9941000000000003e-05 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108394 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.5361243790000003, 2024-09-18T00:59:12.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054381878000000002, 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108394 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5400000000000001e-07 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197088+0000", 2024-09-18T00:59:12.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8830000000000002e-06 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197096+0000", 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8560000000000007e-06 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197101+0000", 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251388+0000", 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054287588999999997 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251427+0000", 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8665999999999999e-05 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251466+0000", 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9369000000000002e-05 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.791 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108395 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.5360989640000007, 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054457976999999998, 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108395 2024-09-18T00:59:12.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197111+0000", 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6000000000000004e-07 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197113+0000", 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4119999999999999e-06 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197119+0000", 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4160000000000001e-06 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197125+0000", 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8320000000000002e-06 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251492+0000", 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054366997 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251529+0000", 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6563000000000002e-05 2024-09-18T00:59:12.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251568+0000", 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9196999999999998e-05 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108396 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.5360758929999996, 2024-09-18T00:59:12.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054537624, 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108396 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:12.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6199999999999999e-07 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197135+0000", 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197139+0000", 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1749999999999998e-06 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197143+0000", 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1350000000000001e-06 2024-09-18T00:59:12.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251594+0000", 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054451160999999998 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251632+0000", 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7840999999999999e-05 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251671+0000", 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9169999999999999e-05 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108413 3.1 3:8453e3d7:::rbd_data.117c8e0d79e4.0000000000000022:20 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.5356318850000008, 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054401959, 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.801 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108413 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197578+0000", 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7199999999999999e-07 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197581+0000", 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8439999999999998e-06 2024-09-18T00:59:12.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197587+0000", 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1759999999999998e-06 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197592+0000", 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4310000000000002e-06 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251903+0000", 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054310418999999999 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251939+0000", 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6310999999999998e-05 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251979+0000", 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0206000000000001e-05 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.805 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108428 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.5354374469999996, 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054482228000000001, 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.806 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108428 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9900000000000001e-07 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197774+0000", 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.373e-06 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197781+0000", 2024-09-18T00:59:12.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3130000000000004e-06 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197786+0000", 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9980000000000003e-06 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252117+0000", 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054331329999999997 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252178+0000", 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0952000000000002e-05 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252254+0000", 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5863000000000002e-05 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108429 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.535416498, 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054624675999999997, 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108429 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197793+0000", 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0600000000000001e-07 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197795+0000", 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.458e-06 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197802+0000", 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.691e-06 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197807+0000", 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2249999999999999e-06 2024-09-18T00:59:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252283+0000", 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054476138 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252348+0000", 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4783999999999995e-05 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252417+0000", 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8973999999999994e-05 2024-09-18T00:59:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108430 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.5353943070000007, 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054759302000000003, 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108430 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.58e-07 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197818+0000", 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5600000000000001e-06 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197824+0000", 2024-09-18T00:59:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.511e-06 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197829+0000", 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1619999999999999e-06 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252446+0000", 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054616988999999998 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252513+0000", 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6558999999999998e-05 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252574+0000", 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1062999999999995e-05 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108540 3.1 3:83363b4a:::rbd_data.117c8e0d79e4.000000000000002d:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.5280149279999993, 2024-09-18T00:59:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056511851000000002, 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108540 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.01e-07 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205197+0000", 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205204+0000", 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4610000000000002e-06 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205210+0000", 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5140000000000001e-06 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.256876+0000", 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051666493000000001 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.256915+0000", 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8724999999999999e-05 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.261706+0000", 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004791086 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.824 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.824 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:12.824 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:12.824 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:12.825 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:12.826 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:12.825+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:12.926 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:12.925+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:12.998 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:12.998 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:13.012 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:13.032 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7] out_osds: [2, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:59:13.032 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:59:13.032 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 4, 5, 6, 7] 2024-09-18T00:59:13.032 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-18T00:59:13.057 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:13.053+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:13.127 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:13.121+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:13.137 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:13.137 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:13.137 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:13.137 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:13.137 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:13.152 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:13.155 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone33 from 0xaa24d64 (overlap 0x28dcff) to 0x0 2024-09-18T00:59:13.228 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:13.225+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:13.241 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-18T00:59:13.263 INFO:teuthology.orchestra.run.smithi042.stdout:4203 read 0x99728c1 thru 0x9982118 (0xf858 bytes) 2024-09-18T00:59:13.270 INFO:teuthology.orchestra.run.smithi042.stdout:4205 read 0x40c40f4 thru 0x40ca797 (0x66a4 bytes) 2024-09-18T00:59:13.272 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.272 INFO:teuthology.orchestra.run.smithi042.stdout:4207 writesame 0x3dbe35b thru 0x3dca1b4 data_size 0x16(0xbe5a bytes) 2024-09-18T00:59:13.283 INFO:teuthology.orchestra.run.smithi042.stdout:4209 punch from 0x73dd9ae to 0x73e46dc, (0x6d2e bytes) 2024-09-18T00:59:13.283 INFO:teuthology.orchestra.run.smithi042.stdout:4210 punch from 0x8d56b1a to 0x8d5bb19, (0x4fff bytes) 2024-09-18T00:59:13.283 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.283 INFO:teuthology.orchestra.run.smithi042.stdout:4215 writesame 0x67ef7d6 thru 0x67f47cf data_size 0x2(0x4ffa bytes) 2024-09-18T00:59:13.296 INFO:teuthology.orchestra.run.smithi042.stdout:4217 write 0x21f5b7 thru 0x222a5d (0x34a7 bytes) 2024-09-18T00:59:13.306 INFO:teuthology.orchestra.run.smithi042.stdout:4218 write 0xb2c199 thru 0xb34df9 (0x8c61 bytes) 2024-09-18T00:59:13.316 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:13.316 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:13.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:13.316 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4041 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.312344781999997, 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037088000000000003, 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4041 2024-09-18T00:59:13.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003005+0000", 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.187e-06 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003011+0000", 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003029+0000", 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8026999999999999e-05 2024-09-18T00:59:13.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003043+0000", 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039484+0000", 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036440895000000001 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039722+0000", 2024-09-18T00:59:13.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000238507 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040092+0000", 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000369818 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:13.321 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.312287158999993, 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:13.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:59:13.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.312228902000001, 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.326 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:13.330 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.312170463000001, 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:59:13.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:59:13.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.312109246000006, 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.335 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.339 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.312018831000003, 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:59:13.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:59:13.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.311947898, 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.344 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:59:13.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.348 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.311888292000006, 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.349 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:59:13.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:13.353 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.311828922000004, 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:59:13.357 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.311767133000004, 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.358 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:59:13.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.311634310000002, 2024-09-18T00:59:13.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:59:13.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:59:13.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.756416833000003, 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T00:59:13.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T00:59:13.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.755916709000005, 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T00:59:13.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T00:59:13.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:13.377 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.755886200999996, 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:13.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.755861535999998, 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.382 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T00:59:13.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.755833003999996, 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T00:59:13.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T00:59:13.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.755499713000006, 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T00:59:13.391 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.395 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.755457862, 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.396 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T00:59:13.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.754420366000005, 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.402 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T00:59:13.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:59:13.406 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 34.102132361000002, 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:13.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T00:59:13.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T00:59:13.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:13.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:13.412 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:13.414 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.414 INFO:teuthology.orchestra.run.smithi042.stdout:4220 punch from 0x5485f76 to 0x5489cf1, (0x3d7b bytes) 2024-09-18T00:59:13.414 INFO:teuthology.orchestra.run.smithi042.stdout:4221 write 0x79f62cb thru 0x7a0271c (0xc452 bytes) 2024-09-18T00:59:13.414 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.415 INFO:teuthology.orchestra.run.smithi042.stdout:4223 punch from 0xbef839 to 0xbf6288, (0x6a4f bytes) 2024-09-18T00:59:13.415 INFO:teuthology.orchestra.run.smithi042.stdout:4224 trunc from 0xaa24d64 to 0x47de2d1 2024-09-18T00:59:13.415 INFO:teuthology.orchestra.run.smithi042.stdout:4226 write 0xb826094 thru 0xb828282 (0x21ef bytes) 2024-09-18T00:59:13.415 INFO:teuthology.orchestra.run.smithi042.stdout:4228 writesame 0x7f8a1b4 thru 0x7f8aa56 data_size 0xb(0x8a3 bytes) 2024-09-18T00:59:13.415 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-18T00:59:13.405+0000 7fdb35eeb640 -1 mon.a@0(leader).osd e102 definitely_dead 0 2024-09-18T00:59:13.415 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:13.416 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:13.413+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:13.418 INFO:teuthology.orchestra.run.smithi042.stdout:4229 trunc from 0xb828283 to 0x78bcfcd 2024-09-18T00:59:13.432 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.452 INFO:teuthology.orchestra.run.smithi042.stdout:4234 write 0xeaab8b0 thru 0xeaadc8f (0x23e0 bytes) 2024-09-18T00:59:13.455 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.455 INFO:teuthology.orchestra.run.smithi042.stdout:4236 read 0xb62cf78 thru 0xb6312d0 (0x4359 bytes) 2024-09-18T00:59:13.456 INFO:teuthology.orchestra.run.smithi042.stdout:4237 write 0x9b0ad5e thru 0x9b134af (0x8752 bytes) 2024-09-18T00:59:13.458 INFO:teuthology.orchestra.run.smithi042.stdout:4238 writesame 0xd47ef99 thru 0xd48665f data_size 0x1(0x76c7 bytes) 2024-09-18T00:59:13.489 INFO:teuthology.orchestra.run.smithi042.stdout:4240 read 0xc03c7c3 thru 0xc04bdd0 (0xf60e bytes) 2024-09-18T00:59:13.489 INFO:teuthology.orchestra.run.smithi042.stdout:4241 write 0x8654d40 thru 0x8659e7f (0x5140 bytes) 2024-09-18T00:59:13.493 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.493 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.493 INFO:teuthology.orchestra.run.smithi042.stdout:4245 writesame 0x9b23ab6 thru 0x9b27e42 data_size 0x1(0x438d bytes) 2024-09-18T00:59:13.512 INFO:teuthology.orchestra.run.smithi042.stdout:4246 write 0x4b49898 thru 0x4b4f7fe (0x5f67 bytes) 2024-09-18T00:59:13.514 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:13.513+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:13.516 INFO:teuthology.orchestra.run.smithi042.stdout:4247 write 0x4f1cdac thru 0x4f29fce (0xd223 bytes) 2024-09-18T00:59:13.519 INFO:teuthology.orchestra.run.smithi042.stdout:4248 write 0x5b5175d thru 0x5b5d853 (0xc0f7 bytes) 2024-09-18T00:59:13.523 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.523 INFO:teuthology.orchestra.run.smithi042.stdout:4250 read 0x98a6894 thru 0x98a7dd6 (0x1543 bytes) 2024-09-18T00:59:13.524 INFO:teuthology.orchestra.run.smithi042.stdout:4251 read 0x511078c thru 0x5112d29 (0x259e bytes) 2024-09-18T00:59:13.525 INFO:teuthology.orchestra.run.smithi042.stdout:4252 write 0xdfde47 thru 0xdfe31f (0x4d9 bytes) 2024-09-18T00:59:13.532 INFO:teuthology.orchestra.run.smithi042.stdout:4254 trunc from 0xeaadc90 to 0xc90c527 2024-09-18T00:59:13.549 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:13.549 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:13.560 INFO:teuthology.orchestra.run.smithi042.stdout:4255 trunc from 0xc90c527 to 0x8f81abe 2024-09-18T00:59:13.568 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:13.580 INFO:teuthology.orchestra.run.smithi042.stdout:4256 read 0xe60858 thru 0xe6ebd9 (0xe382 bytes) 2024-09-18T00:59:13.598 INFO:teuthology.orchestra.run.smithi042.stdout:4257 writesame 0xaa462c5 thru 0xaa4edbd data_size 0x43(0x8af9 bytes) 2024-09-18T00:59:13.601 INFO:teuthology.orchestra.run.smithi042.stdout:4259 write 0x7b54ac9 thru 0x7b63362 (0xe89a bytes) 2024-09-18T00:59:13.605 INFO:teuthology.orchestra.run.smithi042.stdout:4260 write 0x9a22a4d thru 0x9a2ae7e (0x8432 bytes) 2024-09-18T00:59:13.608 INFO:teuthology.orchestra.run.smithi042.stdout:4261 punch from 0x2d65d90 to 0x2d66955, (0xbc5 bytes) 2024-09-18T00:59:13.609 INFO:teuthology.orchestra.run.smithi042.stdout:4262 read 0xa101b2a thru 0xa10438e (0x2865 bytes) 2024-09-18T00:59:13.609 INFO:teuthology.orchestra.run.smithi042.stdout:4263 read 0xa1e1313 thru 0xa1ebce5 (0xa9d3 bytes) 2024-09-18T00:59:13.609 INFO:teuthology.orchestra.run.smithi042.stdout:4264 read 0x43394d2 thru 0x4342ea1 (0x99d0 bytes) 2024-09-18T00:59:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:4265 writesame 0x2625527 thru 0x262e6f5 data_size 0x1(0x91cf bytes) 2024-09-18T00:59:13.615 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:13.609+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout:4266 read 0x1717bf7 thru 0x171fb40 (0x7f4a bytes) 2024-09-18T00:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout:4269 write 0xdcccd0 thru 0xddae5c (0xe18d bytes) 2024-09-18T00:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout:4270 trunc from 0xaa4edbe to 0xa1986ff 2024-09-18T00:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout:4272 write 0x929d983 thru 0x92a43f1 (0x6a6f bytes) 2024-09-18T00:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout:4273 write 0x14748e2 thru 0x1484713 (0xfe32 bytes) 2024-09-18T00:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout:4276 writesame 0xe75df98 thru 0xe761404 data_size 0x1(0x346d bytes) 2024-09-18T00:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout:4277 trunc from 0xe761405 to 0xa235db8 2024-09-18T00:59:13.716 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:13.713+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:13.719 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:13.719 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:13.719 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:13.719 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:13.719 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:13.736 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:13.746 INFO:teuthology.orchestra.run.smithi042.stdout:4278 writesame 0xbe14c55 thru 0xbe1b05b data_size 0x1(0x6407 bytes) 2024-09-18T00:59:13.749 INFO:teuthology.orchestra.run.smithi042.stdout:4280 write 0x20ebde3 thru 0x20ec7d8 (0x9f6 bytes) 2024-09-18T00:59:13.752 INFO:teuthology.orchestra.run.smithi042.stdout:4281 writesame 0x7c46c81 thru 0x7c56c74 data_size 0x2(0xfff4 bytes) 2024-09-18T00:59:13.800 INFO:teuthology.orchestra.run.smithi042.stdout:4282 punch from 0x9625929 to 0x9633322, (0xd9f9 bytes) 2024-09-18T00:59:13.804 INFO:teuthology.orchestra.run.smithi042.stdout:4285 trunc from 0xbe1b05c to 0x93c1470 2024-09-18T00:59:13.817 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:13.813+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:13.822 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.822 INFO:teuthology.orchestra.run.smithi042.stdout:4287 read 0x1ab9ed6 thru 0x1abb1c4 (0x12ef bytes) 2024-09-18T00:59:13.823 INFO:teuthology.orchestra.run.smithi042.stdout:4289 trunc from 0x93c1470 to 0x8eaaac1 2024-09-18T00:59:13.836 INFO:teuthology.orchestra.run.smithi042.stdout:4290 write 0x3214ffa thru 0x32194b3 (0x44ba bytes) 2024-09-18T00:59:13.847 INFO:teuthology.orchestra.run.smithi042.stdout:4291 trunc from 0x8eaaac1 to 0xde7614a 2024-09-18T00:59:13.856 INFO:teuthology.orchestra.run.smithi042.stdout:4292 writesame 0x2e3b98a thru 0x2e42e4f data_size 0x1(0x74c6 bytes) 2024-09-18T00:59:13.891 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:13.891 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:13.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:13.891 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:13.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.891 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:13.891 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:13.891 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.823459557000007, 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:13.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:59:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.930925737999999, 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T00:59:13.897 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T00:59:13.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.902 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1818589480000004, 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1714844920000003, 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T00:59:13.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T00:59:13.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1714118100000004, 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T00:59:13.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.171380944, 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.916 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T00:59:13.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T00:59:13.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T00:59:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T00:59:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T00:59:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1713281699999998, 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:13.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.925 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1713020040000002, 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T00:59:13.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1712768489999998, 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T00:59:13.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1708535380000002, 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.935 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T00:59:13.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T00:59:13.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1708151600000001, 2024-09-18T00:59:13.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T00:59:13.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T00:59:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.170790395, 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T00:59:13.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1639956309999997, 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1639700309999998, 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:13.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T00:59:13.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1639466880000002, 2024-09-18T00:59:13.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T00:59:13.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1639242489999999, 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:13.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T00:59:13.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1639034759999998, 2024-09-18T00:59:13.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T00:59:13.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.16388245, 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T00:59:13.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T00:59:13.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1638418680000004, 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T00:59:13.975 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T00:59:13.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1638077689999999, 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:13.980 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T00:59:13.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T00:59:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T00:59:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:13.984 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:13.984 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:13.984 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:13.986 INFO:teuthology.orchestra.run.smithi042.stdout:4293 writesame 0x9965cdb thru 0x9968c3b data_size 0xd(0x2f61 bytes) 2024-09-18T00:59:13.986 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.986 INFO:teuthology.orchestra.run.smithi042.stdout:4297 read 0xaa0e4d thru 0xaa46b1 (0x3865 bytes) 2024-09-18T00:59:13.986 INFO:teuthology.orchestra.run.smithi042.stdout:4298 read 0x556e945 thru 0x55707f9 (0x1eb5 bytes) 2024-09-18T00:59:13.986 INFO:teuthology.orchestra.run.smithi042.stdout:4299 read 0x4273d95 thru 0x427d9cb (0x9c37 bytes) 2024-09-18T00:59:13.986 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.987 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:13.988 INFO:teuthology.orchestra.run.smithi042.stdout:4301 trunc from 0xde7614a to 0xbf7bb30 2024-09-18T00:59:13.988 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.988 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.988 INFO:teuthology.orchestra.run.smithi042.stdout:4305 write 0x72fc51 thru 0x73cd3e (0xd0ee bytes) 2024-09-18T00:59:13.989 INFO:teuthology.orchestra.run.smithi042.stdout:4308 read 0x7279284 thru 0x7286d33 (0xdab0 bytes) 2024-09-18T00:59:13.989 INFO:teuthology.orchestra.run.smithi042.stdout:4309 trunc from 0xbf7bb30 to 0x6d84cdc 2024-09-18T00:59:13.989 INFO:teuthology.orchestra.run.smithi042.stdout:4310 read 0x429a81e thru 0x42a63a8 (0xbb8b bytes) 2024-09-18T00:59:13.989 INFO:teuthology.orchestra.run.smithi042.stdout:4311 writesame 0xb5f203a thru 0xb5f5a3d data_size 0x4f(0x3a04 bytes) 2024-09-18T00:59:13.989 INFO:teuthology.orchestra.run.smithi042.stdout:4312 read 0x876159f thru 0x876c27d (0xacdf bytes) 2024-09-18T00:59:13.989 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:13.985+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:13.990 INFO:teuthology.orchestra.run.smithi042.stdout:4313 read 0x926e69d thru 0x927df84 (0xf8e8 bytes) 2024-09-18T00:59:13.990 INFO:teuthology.orchestra.run.smithi042.stdout:4315 write 0x6069529 thru 0x607796b (0xe443 bytes) 2024-09-18T00:59:13.991 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:13.991 INFO:teuthology.orchestra.run.smithi042.stdout:4317 read 0x22612b2 thru 0x226a9d9 (0x9728 bytes) 2024-09-18T00:59:13.993 INFO:teuthology.orchestra.run.smithi042.stdout:4320 read 0x4e65d92 thru 0x4e7252f (0xc79e bytes) 2024-09-18T00:59:13.994 INFO:teuthology.orchestra.run.smithi042.stdout:4321 write 0x5da63c9 thru 0x5daf7b1 (0x93e9 bytes) 2024-09-18T00:59:14.003 INFO:teuthology.orchestra.run.smithi042.stdout:4322 trunc from 0xb5f5a3e to 0x88c0224 2024-09-18T00:59:14.020 INFO:teuthology.orchestra.run.smithi042.stdout:4323 read 0x580c3a1 thru 0x581add0 (0xea30 bytes) 2024-09-18T00:59:14.022 INFO:teuthology.orchestra.run.smithi042.stdout:4325 trunc from 0x88c0224 to 0x976dad3 2024-09-18T00:59:14.033 INFO:teuthology.orchestra.run.smithi042.stdout:4326 punch from 0x7a824e7 to 0x7a8250a, (0x23 bytes) 2024-09-18T00:59:14.033 INFO:teuthology.orchestra.run.smithi042.stdout:4327 punch from 0x25bb5e5 to 0x25c5623, (0xa03e bytes) 2024-09-18T00:59:14.033 INFO:teuthology.orchestra.run.smithi042.stdout:4328 writesame 0x2ed543d thru 0x2ee5044 data_size 0x14(0xfc08 bytes) 2024-09-18T00:59:14.039 INFO:teuthology.orchestra.run.smithi042.stdout:4329 read 0x95016ab thru 0x9502ae0 (0x1436 bytes) 2024-09-18T00:59:14.040 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:14.040 INFO:teuthology.orchestra.run.smithi042.stdout:4331 read 0x131a88a thru 0x1328b1d (0xe294 bytes) 2024-09-18T00:59:14.041 INFO:teuthology.orchestra.run.smithi042.stdout:4334 writesame 0xb17b5 thru 0xc01e4 data_size 0x18(0xea30 bytes) 2024-09-18T00:59:14.045 INFO:teuthology.orchestra.run.smithi042.stderr:marked out osd.0. 2024-09-18T00:59:14.050 INFO:teuthology.orchestra.run.smithi042.stdout:4335 writesame 0x851e2fd thru 0x852c882 data_size 0x15(0xe586 bytes) 2024-09-18T00:59:14.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:59:14.067 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:14.087 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:14.085+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:14.136 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:14.136 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:14.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:14.136 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:136619 3.f 3.1890d74f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-18T00:59:14.136 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:14.074696+0000", 2024-09-18T00:59:14.136 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.060761545, 2024-09-18T00:59:14.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060795615999999997, 2024-09-18T00:59:14.136 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:14.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:14.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "waiting for peered", 2024-09-18T00:59:14.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 136619 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:14.074696+0000", 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:14.074696+0000", 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:14.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:14.074699+0000", 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3699999999999999e-06 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:14.074715+0000", 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5744999999999999e-05 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:14.074745+0000", 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0329999999999999e-05 2024-09-18T00:59:14.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:14.074760+0000", 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4837e-05 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:14.074882+0000", 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121836 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:14.074901+0000", 2024-09-18T00:59:14.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8354000000000001e-05 2024-09-18T00:59:14.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:14.140 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:14.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:14.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:14.140 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:14.140 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 1 2024-09-18T00:59:14.140 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:14.151 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:14.151 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:14.152 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:14.170 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:14.188 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:14.185+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:14.190 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:14.190 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:14.209 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:14.230 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:14.251 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:14.272 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:14.285 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:14.290 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:14.285+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:14.295 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:14.389 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:14.389+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:14.411 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:14.411 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:14.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:14.490 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:14.489+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:14.551 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:14.551 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:14.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:14.610 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:14.609+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:14.691 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:14.689+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:14.692 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:14.692 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:14.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:14.792 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:14.789+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:14.878 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:14.879 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:14.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:14.895 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:14.893+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:14.995 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:14.993+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:15.022 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:15.022 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:15.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:15.055 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:59:15.130 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:15.129+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:15.178 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:15.178 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:15.192 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:15.234 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:15.233+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:15.300 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:15.297+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:15.360 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:15.360 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:15.375 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:15.399 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:15.397+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:15.500 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:15.493+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:15.505 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:15.505 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:15.519 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:15.600 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:15.597+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:15.686 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:15.686 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:15.700 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:15.701 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:15.697+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:15.802 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:15.801+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:15.823 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:15.823 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:15.837 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:15.902 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:15.901+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:16.003 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:16.003 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:16.003 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:16.001+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:16.017 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:16.104 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:16.101+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:16.112 INFO:teuthology.orchestra.run.smithi042.stdout:4337 write 0xdbb20b6 thru 0xdbbade1 (0x8d2c bytes) 2024-09-18T00:59:16.117 INFO:teuthology.orchestra.run.smithi042.stdout:4338 trunc from 0xdbbade2 to 0xd6cdf58 2024-09-18T00:59:16.131 INFO:teuthology.orchestra.run.smithi042.stdout:4339 read 0xf5cc0f thru 0xf62c89 (0x607b bytes) 2024-09-18T00:59:16.142 INFO:teuthology.orchestra.run.smithi042.stdout:4341 trunc from 0xd6cdf58 to 0x986cc21 2024-09-18T00:59:16.157 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:4344 trunc from 0x986cc21 to 0xe70c043 2024-09-18T00:59:16.180 INFO:teuthology.orchestra.run.smithi042.stdout:4345 read 0x704c8a thru 0x709d30 (0x50a7 bytes) 2024-09-18T00:59:16.181 INFO:teuthology.orchestra.run.smithi042.stdout:4346 write 0x17f4b44 thru 0x17fcb58 (0x8015 bytes) 2024-09-18T00:59:16.184 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.185 INFO:teuthology.orchestra.run.smithi042.stdout:4349 writesame 0xaace274 thru 0xaada531 data_size 0xe(0xc2be bytes) 2024-09-18T00:59:16.190 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:16.190 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:16.192 INFO:teuthology.orchestra.run.smithi042.stdout:4350 read 0x8fc352 thru 0x9021e4 (0x5e93 bytes) 2024-09-18T00:59:16.205 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:16.201+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:16.205 INFO:teuthology.orchestra.run.smithi042.stdout:4351 trunc from 0xe70c043 to 0x8e1605b 2024-09-18T00:59:16.216 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:16.221 INFO:teuthology.orchestra.run.smithi042.stdout:4352 read 0x6cfc33a thru 0x6cfc384 (0x4b bytes) 2024-09-18T00:59:16.222 INFO:teuthology.orchestra.run.smithi042.stdout:4353 read 0x5ce959 thru 0x5d7f5f (0x9607 bytes) 2024-09-18T00:59:16.223 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.223 INFO:teuthology.orchestra.run.smithi042.stdout:4355 write 0x547267d thru 0x5476597 (0x3f1b bytes) 2024-09-18T00:59:16.226 INFO:teuthology.orchestra.run.smithi042.stdout:4356 write 0x3f5107b thru 0x3f59948 (0x88ce bytes) 2024-09-18T00:59:16.229 INFO:teuthology.orchestra.run.smithi042.stdout:4357 write 0xa4aab thru 0xa6518 (0x1a6e bytes) 2024-09-18T00:59:16.243 INFO:teuthology.orchestra.run.smithi042.stdout:4358 trunc from 0x8e1605b to 0x4e2d0ce 2024-09-18T00:59:16.299 INFO:teuthology.orchestra.run.smithi042.stdout:4359 writesame 0xe49551d thru 0xe49f7af data_size 0x1(0xa293 bytes) 2024-09-18T00:59:16.302 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.305 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:16.305+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:16.310 INFO:teuthology.orchestra.run.smithi042.stdout:4362 trunc from 0xe49f7b0 to 0xab9934b 2024-09-18T00:59:16.315 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:16.339 INFO:teuthology.orchestra.run.smithi042.stdout:4363 trunc from 0xab9934b to 0x5e0a1a7 2024-09-18T00:59:16.352 INFO:teuthology.orchestra.run.smithi042.stdout:4365 write 0x20a64b thru 0x219ad8 (0xf48e bytes) 2024-09-18T00:59:16.373 INFO:teuthology.orchestra.run.smithi042.stdout:4366 write 0xb78230b thru 0xb785114 (0x2e0a bytes) 2024-09-18T00:59:16.375 INFO:teuthology.orchestra.run.smithi042.stdout:4368 write 0x8e7f930 thru 0x8e88c69 (0x933a bytes) 2024-09-18T00:59:16.377 INFO:teuthology.orchestra.run.smithi042.stdout:4370 write 0x3255b24 thru 0x325ac74 (0x5151 bytes) 2024-09-18T00:59:16.380 INFO:teuthology.orchestra.run.smithi042.stdout:4371 write 0x816e6c2 thru 0x8174b16 (0x6455 bytes) 2024-09-18T00:59:16.381 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.381 INFO:teuthology.orchestra.run.smithi042.stdout:4373 read 0x4efb29 thru 0x4ff4ad (0xf985 bytes) 2024-09-18T00:59:16.382 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.382 INFO:teuthology.orchestra.run.smithi042.stdout:4377 punch from 0x6844990 to 0x6847116, (0x2786 bytes) 2024-09-18T00:59:16.382 INFO:teuthology.orchestra.run.smithi042.stdout:4378 punch from 0xa7da87f to 0xa7dcf01, (0x2682 bytes) 2024-09-18T00:59:16.382 INFO:teuthology.orchestra.run.smithi042.stdout:4380 read 0x9b08322 thru 0x9b147a4 (0xc483 bytes) 2024-09-18T00:59:16.399 INFO:teuthology.orchestra.run.smithi042.stdout:4381 trunc from 0xb785115 to 0x18ed900 2024-09-18T00:59:16.406 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:16.405+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:16.419 INFO:teuthology.orchestra.run.smithi042.stdout:4382 read 0x13ca1cb thru 0x13d5773 (0xb5a9 bytes) 2024-09-18T00:59:16.419 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.428 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:16.428 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:16.441 INFO:teuthology.orchestra.run.smithi042.stdout:4384 writesame 0x9f9f5f1 thru 0x9fa1c4a data_size 0x1(0x265a bytes) 2024-09-18T00:59:16.443 INFO:teuthology.orchestra.run.smithi042.stdout:4385 read 0x46d5b8b thru 0x46df39b (0x9811 bytes) 2024-09-18T00:59:16.443 INFO:teuthology.orchestra.run.smithi042.stdout:4387 writesame 0x23bbffa thru 0x23be7eb data_size 0x1(0x27f2 bytes) 2024-09-18T00:59:16.445 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:16.450 INFO:teuthology.orchestra.run.smithi042.stdout:4389 writesame 0x7aa69da thru 0x7aa72e2 data_size 0x3(0x909 bytes) 2024-09-18T00:59:16.453 INFO:teuthology.orchestra.run.smithi042.stdout:4392 read 0x4ca414f thru 0x4caca76 (0x8928 bytes) 2024-09-18T00:59:16.453 INFO:teuthology.orchestra.run.smithi042.stdout:4394 punch from 0x89966b7 to 0x89a334b, (0xcc94 bytes) 2024-09-18T00:59:16.453 INFO:teuthology.orchestra.run.smithi042.stdout:4395 write 0x75df1dc thru 0x75e55d6 (0x63fb bytes) 2024-09-18T00:59:16.455 INFO:teuthology.orchestra.run.smithi042.stdout:4396 write 0x93b1d63 thru 0x93bc585 (0xa823 bytes) 2024-09-18T00:59:16.458 INFO:teuthology.orchestra.run.smithi042.stdout:4397 punch from 0x6c3071 to 0x6c9217, (0x61a6 bytes) 2024-09-18T00:59:16.458 INFO:teuthology.orchestra.run.smithi042.stdout:4399 write 0x27687e9 thru 0x27729bd (0xa1d5 bytes) 2024-09-18T00:59:16.459 INFO:teuthology.orchestra.run.smithi042.stdout:4400 punch from 0x1acdac6 to 0x1acf774, (0x1cae bytes) 2024-09-18T00:59:16.459 INFO:teuthology.orchestra.run.smithi042.stdout:4402 read 0x3bd13cd thru 0x3bdec07 (0xd83b bytes) 2024-09-18T00:59:16.460 INFO:teuthology.orchestra.run.smithi042.stdout:4405 punch from 0x685d59e to 0x686493f, (0x73a1 bytes) 2024-09-18T00:59:16.460 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.460 INFO:teuthology.orchestra.run.smithi042.stdout:4407 writesame 0x9ab8e8c thru 0x9ac1bc9 data_size 0x2(0x8d3e bytes) 2024-09-18T00:59:16.472 INFO:teuthology.orchestra.run.smithi042.stdout:4408 writesame 0x2bcabe9 thru 0x2bd6383 data_size 0x1(0xb79b bytes) 2024-09-18T00:59:16.504 INFO:teuthology.orchestra.run.smithi042.stdout:4410 write 0x66c1246 thru 0x66c9cdd (0x8a98 bytes) 2024-09-18T00:59:16.506 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:16.507 INFO:teuthology.orchestra.run.smithi042.stdout:4412 write 0x9226c37 thru 0x922bcea (0x50b4 bytes) 2024-09-18T00:59:16.508 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:16.505+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:16.509 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.509 INFO:teuthology.orchestra.run.smithi042.stdout:4414 read 0x77d59a9 thru 0x77dec56 (0x92ae bytes) 2024-09-18T00:59:16.509 INFO:teuthology.orchestra.run.smithi042.stdout:4415 punch from 0x5672e77 to 0x567fcb9, (0xce42 bytes) 2024-09-18T00:59:16.509 INFO:teuthology.orchestra.run.smithi042.stdout:4416 write 0x4bee03f thru 0x4bf62d9 (0x829b bytes) 2024-09-18T00:59:16.511 INFO:teuthology.orchestra.run.smithi042.stdout:4419 write 0x362f3e9 thru 0x363cb6d (0xd785 bytes) 2024-09-18T00:59:16.513 INFO:teuthology.orchestra.run.smithi042.stdout:4420 writesame 0x8f2c614 thru 0x8f31151 data_size 0x1(0x4b3e bytes) 2024-09-18T00:59:16.523 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:16.524 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:16.525 INFO:teuthology.orchestra.run.smithi042.stdout:4421 punch from 0x65adb67 to 0x65b058f, (0x2a28 bytes) 2024-09-18T00:59:16.526 INFO:teuthology.orchestra.run.smithi042.stdout:4422 read 0x4ec84dc thru 0x4ed0692 (0x81b7 bytes) 2024-09-18T00:59:16.534 INFO:teuthology.orchestra.run.smithi042.stdout:4424 write 0xa3f14a6 thru 0xa40067d (0xf1d8 bytes) 2024-09-18T00:59:16.536 INFO:teuthology.orchestra.run.smithi042.stdout:4425 writesame 0x1af7060 thru 0x1b06967 data_size 0x34(0xf908 bytes) 2024-09-18T00:59:16.543 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:16.544 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:16.552 INFO:teuthology.orchestra.run.smithi042.stdout:4426 trunc from 0xa40067e to 0x16242dd 2024-09-18T00:59:16.564 INFO:teuthology.orchestra.run.smithi042.stdout:4427 read 0x1173b9f thru 0x1182af3 (0xef55 bytes) 2024-09-18T00:59:16.565 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:16.566 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:16.577 INFO:teuthology.orchestra.run.smithi042.stdout:4429 writesame 0x4873a50 thru 0x48783bd data_size 0x27(0x496e bytes) 2024-09-18T00:59:16.580 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.580 INFO:teuthology.orchestra.run.smithi042.stdout:4434 writesame 0x355837e thru 0x3565fcb data_size 0x2(0xdc4e bytes) 2024-09-18T00:59:16.585 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:16.586 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:16.589 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:16.590 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:16.596 INFO:teuthology.orchestra.run.smithi042.stdout:4435 read 0xc51bce thru 0xc592e8 (0x771b bytes) 2024-09-18T00:59:16.609 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:16.609 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:16.609+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:16.610 INFO:teuthology.orchestra.run.smithi042.stdout:4436 writesame 0x6d8204b thru 0x6d8487b data_size 0x1(0x2831 bytes) 2024-09-18T00:59:16.611 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:16.611 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:16.612 INFO:teuthology.orchestra.run.smithi042.stdout:4437 write 0x454fdfa thru 0x455154e (0x1755 bytes) 2024-09-18T00:59:16.613 INFO:teuthology.orchestra.run.smithi042.stdout:4438 write 0x3b3e5a1 thru 0x3b43d6e (0x57ce bytes) 2024-09-18T00:59:16.614 INFO:teuthology.orchestra.run.smithi042.stdout:4439 punch from 0x25acdb1 to 0x25af90d, (0x2b5c bytes) 2024-09-18T00:59:16.615 INFO:teuthology.orchestra.run.smithi042.stdout:4442 punch from 0x3da2c73 to 0x3db2178, (0xf505 bytes) 2024-09-18T00:59:16.615 INFO:teuthology.orchestra.run.smithi042.stdout:4445 read 0x5ef66db thru 0x5efbb3d (0x5463 bytes) 2024-09-18T00:59:16.615 INFO:teuthology.orchestra.run.smithi042.stdout:4446 read 0x58fd2ae thru 0x590615b (0x8eae bytes) 2024-09-18T00:59:16.633 INFO:teuthology.orchestra.run.smithi042.stdout:4447 writesame 0xd41c1e1 thru 0xd426de3 data_size 0x1(0xac03 bytes) 2024-09-18T00:59:16.634 INFO:teuthology.orchestra.run.smithi042.stdout:4448 write 0x3090ad0 thru 0x3095930 (0x4e61 bytes) 2024-09-18T00:59:16.636 INFO:teuthology.orchestra.run.smithi042.stdout:4449 write 0x63c062f thru 0x63c89c4 (0x8396 bytes) 2024-09-18T00:59:16.637 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:16.637 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:16.637 INFO:teuthology.orchestra.run.smithi042.stdout:4450 writesame 0x842b386 thru 0x8438cae data_size 0x57(0xd929 bytes) 2024-09-18T00:59:16.639 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.639 INFO:teuthology.orchestra.run.smithi042.stdout:4453 writesame 0x4e82e21 thru 0x4e8edb2 data_size 0xd9(0xbf92 bytes) 2024-09-18T00:59:16.641 INFO:teuthology.orchestra.run.smithi042.stdout:4457 writesame 0x2949a2d thru 0x294bc8f data_size 0x1(0x2263 bytes) 2024-09-18T00:59:16.646 INFO:teuthology.orchestra.run.smithi042.stdout:4458 write 0x40bb15e thru 0x40c2fb9 (0x7e5c bytes) 2024-09-18T00:59:16.648 INFO:teuthology.orchestra.run.smithi042.stdout:4459 write 0xcd5e99f thru 0xcd6a61d (0xbc7f bytes) 2024-09-18T00:59:16.650 INFO:teuthology.orchestra.run.smithi042.stdout:4460 writesame 0x7f779da thru 0x7f8454a data_size 0x1(0xcb71 bytes) 2024-09-18T00:59:16.655 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:16.655 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:16.675 INFO:teuthology.orchestra.run.smithi042.stdout:4461 write 0x3b78e6a thru 0x3b7fffa (0x7191 bytes) 2024-09-18T00:59:16.677 INFO:teuthology.orchestra.run.smithi042.stdout:4462 write 0x1f28e50 thru 0x1f31d1a (0x8ecb bytes) 2024-09-18T00:59:16.679 INFO:teuthology.orchestra.run.smithi042.stdout:4465 writesame 0x61181e2 thru 0x611b80c data_size 0x1(0x362b bytes) 2024-09-18T00:59:16.687 INFO:teuthology.orchestra.run.smithi042.stdout:4466 write 0x89cfc9d thru 0x89da870 (0xabd4 bytes) 2024-09-18T00:59:16.689 INFO:teuthology.orchestra.run.smithi042.stdout:4467 punch from 0x99f495a to 0x9a00c3b, (0xc2e1 bytes) 2024-09-18T00:59:16.689 INFO:teuthology.orchestra.run.smithi042.stdout:4468 write 0x8e6476a thru 0x8e65059 (0x8f0 bytes) 2024-09-18T00:59:16.690 INFO:teuthology.orchestra.run.smithi042.stdout:4469 write 0x5de9443 thru 0x5df2474 (0x9032 bytes) 2024-09-18T00:59:16.709 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:16.709+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:16.711 INFO:teuthology.orchestra.run.smithi042.stdout:4472 trunc from 0xd426de4 to 0x110cbda 2024-09-18T00:59:16.743 INFO:teuthology.orchestra.run.smithi042.stdout:4473 write 0x8053a73 thru 0x8054b56 (0x10e4 bytes) 2024-09-18T00:59:16.746 INFO:teuthology.orchestra.run.smithi042.stdout:4477 punch from 0x34cb19b to 0x34d1ef2, (0x6d57 bytes) 2024-09-18T00:59:16.758 INFO:teuthology.orchestra.run.smithi042.stdout:4478 trunc from 0x8054b57 to 0xe5e4d2a 2024-09-18T00:59:16.766 INFO:teuthology.orchestra.run.smithi042.stdout:4479 write 0x4143d49 thru 0x4149a7f (0x5d37 bytes) 2024-09-18T00:59:16.768 INFO:teuthology.orchestra.run.smithi042.stdout:4480 read 0xadb923e thru 0xadbd66e (0x4431 bytes) 2024-09-18T00:59:16.769 INFO:teuthology.orchestra.run.smithi042.stdout:4481 writesame 0xaf8f405 thru 0xaf92465 data_size 0x1(0x3061 bytes) 2024-09-18T00:59:16.775 INFO:teuthology.orchestra.run.smithi042.stdout:4482 write 0xe47bc26 thru 0xe481a34 (0x5e0f bytes) 2024-09-18T00:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout:4483 read 0xd4c8198 thru 0xd4cb001 (0x2e6a bytes) 2024-09-18T00:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout:4485 read 0x4e82f7a thru 0x4e884c4 (0x554b bytes) 2024-09-18T00:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout:4486 read 0xc8eca5c thru 0xc8fbf4c (0xf4f1 bytes) 2024-09-18T00:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout:4490 write 0x9f17bbf thru 0x9f18b4b (0xf8d bytes) 2024-09-18T00:59:16.780 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:16.780 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:16.780 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:16.780 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:16.780 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout:4492 read 0x87bf40 thru 0x87c3ac (0x46d bytes) 2024-09-18T00:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout:4494 writesame 0x1da41d4 thru 0x1da482f data_size 0xb(0x65c bytes) 2024-09-18T00:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout:4495 read 0x76e44dd thru 0x76e5179 (0xc9d bytes) 2024-09-18T00:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout:4497 trunc from 0xe5e4d2a to 0xa1c03b4 2024-09-18T00:59:16.797 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout:4498 read 0x7def851 thru 0x7df4bb5 (0x5365 bytes) 2024-09-18T00:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout:4501 write 0xb131d72 thru 0xb139de6 (0x8075 bytes) 2024-09-18T00:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:4503 punch from 0x386adbc to 0x386e8d0, (0x3b14 bytes) 2024-09-18T00:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:4504 trunc from 0xb139de7 to 0xa3ad4d2 2024-09-18T00:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout:4505 write 0xecaef9 thru 0xed2855 (0x795d bytes) 2024-09-18T00:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.840 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:16.841+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout:4507 write 0xb7c551a thru 0xb7c7c31 (0x2718 bytes) 2024-09-18T00:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout:4508 read 0x5d7c814 thru 0x5d7fed6 (0x36c3 bytes) 2024-09-18T00:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout:4509 read 0x6d04fd7 thru 0x6d142ff (0xf329 bytes) 2024-09-18T00:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout:4511 read 0x391964 thru 0x397d12 (0x63af bytes) 2024-09-18T00:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout:4514 read 0x72f4973 thru 0x72fdbf0 (0x927e bytes) 2024-09-18T00:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout:4515 read 0x3af3d2d thru 0x3af4544 (0x818 bytes) 2024-09-18T00:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout:4516 punch from 0x246b4a4 to 0x247814b, (0xcca7 bytes) 2024-09-18T00:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout:4517 read 0x625e82e thru 0x62665e1 (0x7db4 bytes) 2024-09-18T00:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout:4520 writesame 0xca30a12 thru 0xca3a39e data_size 0x1(0x998d bytes) 2024-09-18T00:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.866 INFO:teuthology.orchestra.run.smithi042.stdout:4524 trunc from 0xca3a39f to 0xe765525 2024-09-18T00:59:16.876 INFO:teuthology.orchestra.run.smithi042.stdout:4525 punch from 0xcface46 to 0xcfbb9ca, (0xeb84 bytes) 2024-09-18T00:59:16.876 INFO:teuthology.orchestra.run.smithi042.stdout:4526 read 0x9fdc658 thru 0x9fde3ee (0x1d97 bytes) 2024-09-18T00:59:16.877 INFO:teuthology.orchestra.run.smithi042.stdout:4527 punch from 0xb90d779 to 0xb912411, (0x4c98 bytes) 2024-09-18T00:59:16.877 INFO:teuthology.orchestra.run.smithi042.stdout:4528 read 0xbe666e thru 0xbe76c0 (0x1053 bytes) 2024-09-18T00:59:16.877 INFO:teuthology.orchestra.run.smithi042.stdout:4529 read 0x967e35f thru 0x9686901 (0x85a3 bytes) 2024-09-18T00:59:16.878 INFO:teuthology.orchestra.run.smithi042.stdout:4532 punch from 0x6571c8e to 0x6571ffb, (0x36d bytes) 2024-09-18T00:59:16.879 INFO:teuthology.orchestra.run.smithi042.stdout:4534 writesame 0x4079941 thru 0x4087f4b data_size 0x2f(0xe60b bytes) 2024-09-18T00:59:16.881 INFO:teuthology.orchestra.run.smithi042.stdout:4535 writesame 0x947b5ee thru 0x9489ae3 data_size 0x3(0xe4f6 bytes) 2024-09-18T00:59:16.897 INFO:teuthology.orchestra.run.smithi042.stdout:4537 writesame 0x9092614 thru 0x90a2518 data_size 0xb(0xff05 bytes) 2024-09-18T00:59:16.905 INFO:teuthology.orchestra.run.smithi042.stdout:4538 write 0x49b94a6 thru 0x49c8d81 (0xf8dc bytes) 2024-09-18T00:59:16.912 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:16.909+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:16.917 INFO:teuthology.orchestra.run.smithi042.stdout:4539 read 0xe6e5c8f thru 0xe6ed0b3 (0x7425 bytes) 2024-09-18T00:59:16.918 INFO:teuthology.orchestra.run.smithi042.stdout:4540 writesame 0x437ad33 thru 0x438866b data_size 0x1(0xd939 bytes) 2024-09-18T00:59:16.949 INFO:teuthology.orchestra.run.smithi042.stdout:4541 write 0x27c961a thru 0x27d6f7f (0xd966 bytes) 2024-09-18T00:59:16.951 INFO:teuthology.orchestra.run.smithi042.stdout:4542 read 0x2c39473 thru 0x2c46c2e (0xd7bc bytes) 2024-09-18T00:59:16.952 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:16.969 INFO:teuthology.orchestra.run.smithi042.stdout:4545 trunc from 0xe765525 to 0x7dafda7 2024-09-18T00:59:16.978 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:16.978 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:16.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:16.978 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:16.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.978 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T00:59:16.978 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:16.978 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 87.887904809999995, 2024-09-18T00:59:16.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T00:59:16.978 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:16.978 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T00:59:16.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T00:59:16.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:16.982 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 87.497009859000002, 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:16.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:16.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78518 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1183744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.145479029999997, 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054532499999999998, 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:16.987 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78518 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831040+0000", 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3099999999999997e-07 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831042+0000", 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2879999999999999e-06 2024-09-18T00:59:16.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831050+0000", 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.464e-06 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831055+0000", 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0819999999999998e-06 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:16.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885354+0000", 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298739999999998 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885428+0000", 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4567999999999995e-05 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885572+0000", 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014372699999999999 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:16.991 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78519 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1204224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.145457180999998, 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054707261, 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:16.992 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78519 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831062+0000", 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831064+0000", 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0360000000000001e-06 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831069+0000", 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4149999999999998e-06 2024-09-18T00:59:16.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831075+0000", 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2249999999999997e-06 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885602+0000", 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054526893 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885673+0000", 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0865000000000007e-05 2024-09-18T00:59:16.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885768+0000", 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5422999999999998e-05 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78520 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1224704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.145436725000003, 2024-09-18T00:59:16.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054906695999999998, 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78520 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831084+0000", 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2960000000000001e-06 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831091+0000", 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831095+0000", 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.792e-06 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885802+0000", 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054706795000000003 2024-09-18T00:59:16.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885884+0000", 2024-09-18T00:59:17.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2297000000000002e-05 2024-09-18T00:59:17.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885988+0000", 2024-09-18T00:59:17.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010393199999999999 2024-09-18T00:59:17.000 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78521 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.145391852000003, 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055066616999999998, 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.001 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78521 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831127+0000", 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.99e-07 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831130+0000", 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6819999999999999e-06 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831137+0000", 2024-09-18T00:59:17.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3849999999999996e-06 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831145+0000", 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7360000000000001e-06 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886020+0000", 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054874700999999998 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886096+0000", 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6234000000000002e-05 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886193+0000", 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7280000000000004e-05 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78522 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:17.005 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.145366091, 2024-09-18T00:59:17.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055137938999999997, 2024-09-18T00:59:17.006 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.006 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.006 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:17.006 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.006 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.006 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.006 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78522 2024-09-18T00:59:17.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.006 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831153+0000", 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.37e-07 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831155+0000", 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4420000000000001e-06 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831161+0000", 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831166+0000", 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8749999999999999e-06 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886279+0000", 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055112337999999997 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886290+0000", 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1527e-05 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.009 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78523 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.145345411000001, 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055171425000000003, 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.010 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78523 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0900000000000002e-07 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831176+0000", 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4830000000000002e-06 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831183+0000", 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0679999999999999e-06 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831189+0000", 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886331+0000", 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055142669999999998 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886344+0000", 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2918999999999999e-05 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78524 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.145323294999997, 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055194430000000003, 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:17.014 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78524 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831196+0000", 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1200000000000003e-07 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831198+0000", 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831204+0000", 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2740000000000004e-06 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831209+0000", 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0900000000000004e-06 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886381+0000", 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055172142 2024-09-18T00:59:17.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886389+0000", 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2150000000000001e-06 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78525 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.145302782000002, 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055206535000000001, 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.018 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78525 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:17.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832358+0000", 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001142531 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832364+0000", 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832375+0000", 2024-09-18T00:59:17.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1381e-05 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832382+0000", 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9800000000000001e-06 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886415+0000", 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032678000000001 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886422+0000", 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4449999999999999e-06 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108130 3.1 3:83363b4a:::rbd_data.117c8e0d79e4.000000000000002d:20 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.782587712, 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054351575999999999, 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.022 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108130 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1100000000000001e-07 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193933+0000", 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3690000000000001e-06 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193940+0000", 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1460000000000001e-06 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193944+0000", 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9230000000000001e-06 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.245560+0000", 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051615019999999998 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.245633+0000", 2024-09-18T00:59:17.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3490000000000003e-05 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.248282+0000", 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026492170000000002 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108393 3.1 3:86b5547c:::rbd_data.117c8e0d79e4.0000000000000037:20 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.779496157000001, 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054339392, 2024-09-18T00:59:17.026 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108393 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5499999999999999e-07 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197024+0000", 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.308e-06 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197029+0000", 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197032+0000", 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.994e-06 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251285+0000", 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054253006999999999 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251322+0000", 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7104000000000003e-05 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251362+0000", 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9941000000000003e-05 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108394 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:17.030 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.779433750999999, 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054381878000000002, 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108394 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:17.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5400000000000001e-07 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197088+0000", 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8830000000000002e-06 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197096+0000", 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8560000000000007e-06 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197101+0000", 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:17.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251388+0000", 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054287588999999997 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251427+0000", 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8665999999999999e-05 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251466+0000", 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9369000000000002e-05 2024-09-18T00:59:17.034 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108395 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.779408335999999, 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054457976999999998, 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.035 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108395 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197111+0000", 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6000000000000004e-07 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197113+0000", 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4119999999999999e-06 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197119+0000", 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4160000000000001e-06 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197125+0000", 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8320000000000002e-06 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251492+0000", 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054366997 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251529+0000", 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6563000000000002e-05 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251568+0000", 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9196999999999998e-05 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108396 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.779385265, 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054537624, 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.039 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108396 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:17.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6199999999999999e-07 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197135+0000", 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197139+0000", 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1749999999999998e-06 2024-09-18T00:59:17.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197143+0000", 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1350000000000001e-06 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251594+0000", 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054451160999999998 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251632+0000", 2024-09-18T00:59:17.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7840999999999999e-05 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251671+0000", 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9169999999999999e-05 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108413 3.1 3:8453e3d7:::rbd_data.117c8e0d79e4.0000000000000022:20 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.778941257, 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054401959, 2024-09-18T00:59:17.043 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108413 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197578+0000", 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7199999999999999e-07 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197581+0000", 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8439999999999998e-06 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197587+0000", 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1759999999999998e-06 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197592+0000", 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4310000000000002e-06 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251903+0000", 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054310418999999999 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251939+0000", 2024-09-18T00:59:17.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6310999999999998e-05 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251979+0000", 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0206000000000001e-05 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108428 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.778746819, 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054482228000000001, 2024-09-18T00:59:17.047 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108428 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9900000000000001e-07 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197774+0000", 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.373e-06 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197781+0000", 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3130000000000004e-06 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197786+0000", 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9980000000000003e-06 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252117+0000", 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054331329999999997 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252178+0000", 2024-09-18T00:59:17.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0952000000000002e-05 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252254+0000", 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5863000000000002e-05 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108429 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.778725870000001, 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054624675999999997, 2024-09-18T00:59:17.051 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108429 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197793+0000", 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0600000000000001e-07 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197795+0000", 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.458e-06 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197802+0000", 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.691e-06 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197807+0000", 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2249999999999999e-06 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252283+0000", 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054476138 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252348+0000", 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4783999999999995e-05 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252417+0000", 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8973999999999994e-05 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108430 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:17.055 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.778703678999999, 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054759302000000003, 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108430 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.58e-07 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197818+0000", 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5600000000000001e-06 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197824+0000", 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.511e-06 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197829+0000", 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1619999999999999e-06 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252446+0000", 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054616988999999998 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252513+0000", 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6558999999999998e-05 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252574+0000", 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1062999999999995e-05 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.059 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108540 3.1 3:83363b4a:::rbd_data.117c8e0d79e4.000000000000002d:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.7713243, 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056511851000000002, 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108540 2024-09-18T00:59:17.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:17.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.01e-07 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205197+0000", 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205204+0000", 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4610000000000002e-06 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205210+0000", 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5140000000000001e-06 2024-09-18T00:59:17.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.256876+0000", 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051666493000000001 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.256915+0000", 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8724999999999999e-05 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.261706+0000", 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004791086 2024-09-18T00:59:17.063 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.064 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.064 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.064 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.064 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.064 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:17.065 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:17.065 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:17.061+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:17.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:59:17.166 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:17.165+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:17.232 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:17.233 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:17.248 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:17.292 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:17.293+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:17.368 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:17.365+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:17.373 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:17.373 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:17.373 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:17.373 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:17.373 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:17.386 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:17.473 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:17.473+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:17.549 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:17.549 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:17.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:17.549 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:17.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.549 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4041 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:17.549 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:17.549 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.543392791000002, 2024-09-18T00:59:17.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037088000000000003, 2024-09-18T00:59:17.549 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4041 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003004+0000", 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003005+0000", 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.187e-06 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003011+0000", 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003029+0000", 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8026999999999999e-05 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003043+0000", 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039484+0000", 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036440895000000001 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.039722+0000", 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000238507 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040092+0000", 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000369818 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:17.553 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.543335167999999, 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:59:17.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.557 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.543276911000007, 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:59:17.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:59:17.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:59:17.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:59:17.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.543218472000007, 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.562 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:59:17.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.543157254999997, 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:59:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.543066839999994, 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.571 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:59:17.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.542995907000005, 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.576 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:59:17.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.542936300999997, 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.580 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.584 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.542876930999995, 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:59:17.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:59:17.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.588 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.542815141999995, 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:17.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.592 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.542682318999994, 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.593 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:59:17.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.987464841999994, 2024-09-18T00:59:17.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T00:59:17.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T00:59:17.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:17.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.986964717999996, 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.603 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T00:59:17.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T00:59:17.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.986934210000001, 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T00:59:17.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T00:59:17.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:17.612 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.986909545000003, 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:17.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T00:59:17.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T00:59:17.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.986881013000001, 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.617 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T00:59:17.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T00:59:17.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.986547721999997, 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.621 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T00:59:17.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T00:59:17.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:17.625 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.986505871000006, 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T00:59:17.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T00:59:17.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 88.985468374999996, 2024-09-18T00:59:17.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T00:59:17.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T00:59:17.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.635 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.333180370000001, 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:17.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:17.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T00:59:17.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T00:59:17.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:17.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T00:59:17.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T00:59:17.641 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.641 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.641 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.641 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:17.641 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:17.641 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:17.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:17.643 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:17.641+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:17.743 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:17.741+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:17.787 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:17.844 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:17.841+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:17.904 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:59:17.944 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:17.941+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:17.953 INFO:teuthology.orchestra.run.smithi042.stdout:4546 write 0xbebb18f thru 0xbec34ee (0x8360 bytes) 2024-09-18T00:59:17.957 INFO:teuthology.orchestra.run.smithi042.stdout:4547 read 0x1b907a3 thru 0x1b9a88e (0xa0ec bytes) 2024-09-18T00:59:17.958 INFO:teuthology.orchestra.run.smithi042.stdout:4549 write 0x54fe8af thru 0x5509912 (0xb064 bytes) 2024-09-18T00:59:17.974 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:17.974 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:17.974 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:17.974 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:17.974 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:17.988 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:18.045 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:18.045+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:18.146 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:18.145+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:18.157 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:18.157 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:18.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:18.157 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:18.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.157 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.157 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:18.157 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.088704707000005, 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:59:18.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:59:18.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:18.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:59:18.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:59:18.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:59:18.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:59:18.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.164 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T00:59:18.164 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:18.164 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.196170888000001, 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:18.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T00:59:18.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4471040980000005, 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T00:59:18.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4367296419999995, 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.175 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T00:59:18.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.175 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T00:59:18.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T00:59:18.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4366569600000005, 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:18.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.183 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4366260939999993, 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:59:18.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4365733200000008, 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.188 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.192 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4365471539999994, 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:18.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:18.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:18.197 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.436521999, 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:18.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T00:59:18.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4360986879999995, 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.202 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:18.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T00:59:18.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T00:59:18.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T00:59:18.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T00:59:18.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4360603100000002, 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.207 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T00:59:18.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.208 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:18.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:18.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T00:59:18.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4360355449999993, 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.213 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T00:59:18.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.213 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:18.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.216 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4292407810000007, 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:18.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T00:59:18.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T00:59:18.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T00:59:18.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.221 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.221 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.429215181, 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:18.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4291918379999995, 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:18.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T00:59:18.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T00:59:18.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:18.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T00:59:18.230 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4291693989999992, 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.231 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T00:59:18.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:18.235 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4291486259999999, 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:18.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4291275999999993, 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4290870180000006, 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.244 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4290529190000001, 2024-09-18T00:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T00:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T00:59:18.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.250 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:18.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:18.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.253 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:18.255 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:18.255 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:18.253+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:18.356 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:18.353+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:18.432 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137104 3.1d 3.c090ea1d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:17.958602+0000", 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.47337756800000003, 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.47342382399999999, 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "waiting for peered", 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137104 2024-09-18T00:59:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.958602+0000", 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.958602+0000", 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.958604+0000", 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.291e-06 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.958616+0000", 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2142e-05 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.958635+0000", 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8879e-05 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.958649+0000", 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3743e-05 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.958732+0000", 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3139000000000005e-05 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.958746+0000", 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4314000000000001e-05 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 1 2024-09-18T00:59:18.437 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:18.447 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:18.457 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:18.457+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:18.557 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:18.553+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:18.566 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:18.567 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:18.581 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:18.658 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:18.657+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:18.674 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:18.757 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 115.46598234299999, 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:59:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:59:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.269641289999996, 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.269619852999995, 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:59:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.269524525999998, 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:59:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:59:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:59:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:59:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:59:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:59:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:59:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:59:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.269499087, 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.269476564000001, 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.269453123999995, 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:59:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.269429314000007, 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:59:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.269406949, 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:59:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:59:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.269380331999997, 2024-09-18T00:59:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:59:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:59:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:59:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:59:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:59:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:59:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.265228561000001, 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:59:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:59:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:59:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:59:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:59:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.265207817999993, 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.265162253, 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:59:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.265135912000005, 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:59:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.265113241999998, 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.265094868999995, 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.265075081000006, 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:59:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.265048211000007, 2024-09-18T00:59:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T00:59:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.188857742000003, 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T00:59:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T00:59:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T00:59:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T00:59:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T00:59:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T00:59:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.777792646, 2024-09-18T00:59:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T00:59:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T00:59:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T00:59:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T00:59:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:18.850 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:18.852 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:18.852 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:18.852 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:18.853 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:18.853 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:18.854 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:18.849+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:18.864 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:18.864 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:18.882 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:18.882 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:18.903 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:18.903 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:18.924 INFO:teuthology.orchestra.run.smithi042.stdout:4550 writesame 0x7d8071f thru 0x7d8f465 data_size 0x17(0xed47 bytes) 2024-09-18T00:59:18.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:18.929 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:18.944 INFO:teuthology.orchestra.run.smithi042.stdout:4551 trunc from 0xbec34ef to 0x30494e 2024-09-18T00:59:18.953 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:18.953+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:18.954 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:18.954 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:18.986 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:18.986 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:18.990 INFO:teuthology.orchestra.run.smithi042.stdout:4552 write 0x42a750 thru 0x435d3c (0xb5ed bytes) 2024-09-18T00:59:19.001 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:19.025 INFO:teuthology.orchestra.run.smithi042.stdout:4554 writesame 0xa2f1497 thru 0xa3008db data_size 0x1(0xf445 bytes) 2024-09-18T00:59:19.025 INFO:teuthology.orchestra.run.smithi042.stdout:4555 punch from 0x971df0a to 0x972d4d8, (0xf5ce bytes) 2024-09-18T00:59:19.042 INFO:teuthology.orchestra.run.smithi042.stdout:4557 write 0xea9678e thru 0xeaa1568 (0xaddb bytes) 2024-09-18T00:59:19.046 INFO:teuthology.orchestra.run.smithi042.stdout:4558 write 0xdb97d40 thru 0xdb9b528 (0x37e9 bytes) 2024-09-18T00:59:19.051 INFO:teuthology.orchestra.run.smithi042.stdout:4559 write 0xad38875 thru 0xad3e550 (0x5cdc bytes) 2024-09-18T00:59:19.053 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:19.049+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:19.058 INFO:teuthology.orchestra.run.smithi042.stdout:4560 writesame 0xadd7216 thru 0xadd850b data_size 0x3(0x12f6 bytes) 2024-09-18T00:59:19.062 INFO:teuthology.orchestra.run.smithi042.stdout:4562 punch from 0x5c753e6 to 0x5c82dc8, (0xd9e2 bytes) 2024-09-18T00:59:19.062 INFO:teuthology.orchestra.run.smithi042.stdout:4563 read 0xaccf039 thru 0xacd5dce (0x6d96 bytes) 2024-09-18T00:59:19.062 INFO:teuthology.orchestra.run.smithi042.stdout:4565 write 0xd94e7ba thru 0xd951da8 (0x35ef bytes) 2024-09-18T00:59:19.065 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:19.065 INFO:teuthology.orchestra.run.smithi042.stdout:4567 read 0xa4be196 thru 0xa4cda4c (0xf8b7 bytes) 2024-09-18T00:59:19.065 INFO:teuthology.orchestra.run.smithi042.stdout:4568 write 0x5df8475 thru 0x5dfe601 (0x618d bytes) 2024-09-18T00:59:19.072 INFO:teuthology.orchestra.run.smithi042.stdout:4569 read 0x1d51a51 thru 0x1d5bf39 (0xa4e9 bytes) 2024-09-18T00:59:19.072 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:19.072 INFO:teuthology.orchestra.run.smithi042.stdout:4572 write 0xc00368 thru 0xc0da1e (0xd6b7 bytes) 2024-09-18T00:59:19.076 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6, 7] out_osds: [2, 3, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:59:19.077 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:59:19.077 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T00:59:19.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T00:59:19.096 INFO:teuthology.orchestra.run.smithi042.stdout:4574 trunc from 0xeaa1569 to 0x1ff5f9f 2024-09-18T00:59:19.137 INFO:teuthology.orchestra.run.smithi042.stdout:4577 write 0xad220cd thru 0xad312e2 (0xf216 bytes) 2024-09-18T00:59:19.153 INFO:teuthology.orchestra.run.smithi042.stdout:4578 write 0xcea3f78 thru 0xceae75b (0xa7e4 bytes) 2024-09-18T00:59:19.154 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:19.153+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:19.164 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:19.165 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:19.173 INFO:teuthology.orchestra.run.smithi042.stdout:4579 trunc from 0xceae75c to 0xf0abe4 2024-09-18T00:59:19.187 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:19.212 INFO:teuthology.orchestra.run.smithi042.stdout:4584 writesame 0x84fccab thru 0x85006da data_size 0x4c(0x3a30 bytes) 2024-09-18T00:59:19.215 INFO:teuthology.orchestra.run.smithi042.stdout:4585 punch from 0x744803 to 0x751d96, (0xd593 bytes) 2024-09-18T00:59:19.215 INFO:teuthology.orchestra.run.smithi042.stdout:4586 write 0x2f1b703 thru 0x2f1f5a0 (0x3e9e bytes) 2024-09-18T00:59:19.219 INFO:teuthology.orchestra.run.smithi042.stdout:4587 read 0x26a3379 thru 0x26abfaf (0x8c37 bytes) 2024-09-18T00:59:19.232 INFO:teuthology.orchestra.run.smithi042.stdout:4588 write 0x9d3f926 thru 0x9d418d7 (0x1fb2 bytes) 2024-09-18T00:59:19.236 INFO:teuthology.orchestra.run.smithi042.stdout:4589 read 0x19f8b29 thru 0x19feaf2 (0x5fca bytes) 2024-09-18T00:59:19.236 INFO:teuthology.orchestra.run.smithi042.stdout:4590 read 0x257da6c thru 0x257eb22 (0x10b7 bytes) 2024-09-18T00:59:19.254 INFO:teuthology.orchestra.run.smithi042.stdout:4592 write 0xcb8638b thru 0xcb8e22d (0x7ea3 bytes) 2024-09-18T00:59:19.254 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:19.253+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:19.257 INFO:teuthology.orchestra.run.smithi042.stdout:4593 punch from 0x6d65a40 to 0x6d668e2, (0xea2 bytes) 2024-09-18T00:59:19.257 INFO:teuthology.orchestra.run.smithi042.stdout:4594 read 0x61c9057 thru 0x61cb49a (0x2444 bytes) 2024-09-18T00:59:19.271 INFO:teuthology.orchestra.run.smithi042.stdout:4595 trunc from 0xcb8e22e to 0x395a816 2024-09-18T00:59:19.302 INFO:teuthology.orchestra.run.smithi042.stdout:4596 write 0x4cd3e43 thru 0x4cd84ff (0x46bd bytes) 2024-09-18T00:59:19.305 INFO:teuthology.orchestra.run.smithi042.stdout:4597 punch from 0x2f08950 to 0x2f145f4, (0xbca4 bytes) 2024-09-18T00:59:19.305 INFO:teuthology.orchestra.run.smithi042.stdout:4598 clone 35 order 24 su 524288 sc 9 2024-09-18T00:59:19.332 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:19.332 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:19.344 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:19.355 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:19.353+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 108, 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "82a96d76-ba83-49e2-a985-84604bd599ba", 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-18T00:57:07.138732+0000", 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-18T00:59:18.904312+0000", 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-18T00:57:20.258605+0000", 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-18T00:59:14.038311+0000", 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5804032, 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-18T00:59:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "luminous", 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-18T00:59:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:21.962521+0000", 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-18T00:59:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-18T00:59:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-18T00:59:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-18T00:59:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-18T00:59:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "19", 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-18T00:59:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-18T00:59:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-18T00:59:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 8, 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 8, 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.25, 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:25.721733+0000", 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "23", 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 23, 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 2.9800000190734863, 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 2.9800000190734863, 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.62999999523162842, 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:35.599413+0000", 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "102", 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 36, 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 102, 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T00:59:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 2.7300000190734863, 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 2.7300000190734863, 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.62999999523162842, 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.7200000286102295, 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.7200000286102295, 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T00:59:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:59:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "3f359ce5-6f6e-4161-8554-8ac028347274", 2024-09-18T00:59:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 95, 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/199763443", 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/199763443", 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/199763443", 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/199763443", 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "48687611-1556-4b6d-86af-7872494a91d7", 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 107, 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-18T00:59:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-18T00:59:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/345149800", 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/345149800", 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/345149800", 2024-09-18T00:59:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/345149800", 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "9e6b1f90-ce9b-46d4-92fe-d457f7e415d5", 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-18T00:59:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/431239931", 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/431239931", 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/431239931", 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/431239931", 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:59:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5", 2024-09-18T00:59:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 89, 2024-09-18T00:59:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/1775474180", 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/1775474180", 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/1775474180", 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/1775474180", 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "802ccbeb-7351-4b38-832a-695117c7e493", 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 107, 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6808", 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6809", 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6810", 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6811", 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6814", 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6815", 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6812", 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6813", 2024-09-18T00:59:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6809/4198401207", 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6811/4198401207", 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6815/4198401207", 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6813/4198401207", 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d958b3da-efa1-4d86-b81d-e78874441437", 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 106, 2024-09-18T00:59:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6824", 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6825", 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6826", 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6827", 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6830", 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6831", 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6828", 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6829", 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6825/2599825823", 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6827/2599825823", 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6831/2599825823", 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6829/2599825823", 2024-09-18T00:59:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "51b3cea8-c686-4fd3-b3f4-ba936cf26b64", 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 106, 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6800", 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6801", 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6802", 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6803", 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6806", 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6807", 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6804", 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6805", 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6801/3550656451", 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6803/3550656451", 2024-09-18T00:59:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6807/3550656451", 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6805/3550656451", 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b02ead1c-837a-4b07-9ad1-4e903ed5bc4e", 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-18T00:59:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 106, 2024-09-18T00:59:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6816", 2024-09-18T00:59:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6817", 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6818", 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6819", 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6822", 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6823", 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6820", 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6821", 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6817/2993352359", 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6819/2993352359", 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6823/2993352359", 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6821/2993352359", 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.896477+0000", 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.941756+0000", 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.560180+0000", 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.077394+0000", 2024-09-18T00:59:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.996264+0000", 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.511686+0000", 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.411051+0000", 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.888092+0000", 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [ 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "2.1", 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: 5, 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: 3 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-18T00:59:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [], 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-18T00:59:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-18T00:59:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-18T00:59:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-18T00:59:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-18T00:59:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-18T00:59:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:19.476 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:19.478 INFO:tasks.thrashosds.thrasher:j is {'epoch': 108, 'fsid': '82a96d76-ba83-49e2-a985-84604bd599ba', 'created': '2024-09-18T00:57:07.138732+0000', 'modified': '2024-09-18T00:59:18.904312+0000', 'last_up_change': '2024-09-18T00:57:20.258605+0000', 'last_in_change': '2024-09-18T00:59:14.038311+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T00:57:21.962521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T00:57:25.721733+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.9800000190734863, 'score_stable': 2.9800000190734863, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T00:57:35.599413+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 36, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.7300000190734863, 'score_stable': 2.7300000190734863, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 1.7200000286102295, 'raw_score_stable': 1.7200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3f359ce5-6f6e-4161-8554-8ac028347274', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 199763443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 199763443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 199763443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 199763443}]}, 'public_addr': '172.21.15.42:6801/199763443', 'cluster_addr': '172.21.15.42:6803/199763443', 'heartbeat_back_addr': '172.21.15.42:6807/199763443', 'heartbeat_front_addr': '172.21.15.42:6805/199763443', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '48687611-1556-4b6d-86af-7872494a91d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 107, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 345149800}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 345149800}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 345149800}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 345149800}]}, 'public_addr': '172.21.15.42:6809/345149800', 'cluster_addr': '172.21.15.42:6811/345149800', 'heartbeat_back_addr': '172.21.15.42:6815/345149800', 'heartbeat_front_addr': '172.21.15.42:6813/345149800', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9e6b1f90-ce9b-46d4-92fe-d457f7e415d5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 431239931}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 431239931}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 431239931}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 431239931}]}, 'public_addr': '172.21.15.42:6825/431239931', 'cluster_addr': '172.21.15.42:6827/431239931', 'heartbeat_back_addr': '172.21.15.42:6831/431239931', 'heartbeat_front_addr': '172.21.15.42:6829/431239931', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 89, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 1775474180}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 1775474180}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 1775474180}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 1775474180}]}, 'public_addr': '172.21.15.42:6817/1775474180', 'cluster_addr': '172.21.15.42:6819/1775474180', 'heartbeat_back_addr': '172.21.15.42:6823/1775474180', 'heartbeat_front_addr': '172.21.15.42:6821/1775474180', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '802ccbeb-7351-4b38-832a-695117c7e493', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 107, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6809', 'nonce': 4198401207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6811', 'nonce': 4198401207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6815', 'nonce': 4198401207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6813', 'nonce': 4198401207}]}, 'public_addr': '172.21.15.195:6809/4198401207', 'cluster_addr': '172.21.15.195:6811/4198401207', 'heartbeat_back_addr': '172.21.15.195:6815/4198401207', 'heartbeat_front_addr': '172.21.15.195:6813/4198401207', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd958b3da-efa1-4d86-b81d-e78874441437', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6824', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6825', 'nonce': 2599825823}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6826', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6827', 'nonce': 2599825823}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6830', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6831', 'nonce': 2599825823}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6828', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6829', 'nonce': 2599825823}]}, 'public_addr': '172.21.15.195:6825/2599825823', 'cluster_addr': '172.21.15.195:6827/2599825823', 'heartbeat_back_addr': '172.21.15.195:6831/2599825823', 'heartbeat_front_addr': '172.21.15.195:6829/2599825823', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '51b3cea8-c686-4fd3-b3f4-ba936cf26b64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6800', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6801', 'nonce': 3550656451}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6803', 'nonce': 3550656451}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6807', 'nonce': 3550656451}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6805', 'nonce': 3550656451}]}, 'public_addr': '172.21.15.195:6801/3550656451', 'cluster_addr': '172.21.15.195:6803/3550656451', 'heartbeat_back_addr': '172.21.15.195:6807/3550656451', 'heartbeat_front_addr': '172.21.15.195:6805/3550656451', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b02ead1c-837a-4b07-9ad1-4e903ed5bc4e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6817', 'nonce': 2993352359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6818', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6819', 'nonce': 2993352359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6822', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6823', 'nonce': 2993352359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6820', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6821', 'nonce': 2993352359}]}, 'public_addr': '172.21.15.195:6817/2993352359', 'cluster_addr': '172.21.15.195:6819/2993352359', 'heartbeat_back_addr': '172.21.15.195:6823/2993352359', 'heartbeat_front_addr': '172.21.15.195:6821/2993352359', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.896477+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.941756+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.560180+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.077394+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.996264+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.511686+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.411051+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.888092+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.1', 'osds': [5, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T00:59:19.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:59:19.478 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:19.473+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:19.513 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:19.513 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:19.528 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:19.618 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:19.617+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:19.678 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:19.677+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:19.695 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:19.696 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:19.711 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:19.806 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:19.805+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:19.816 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:59:19.817 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T00:59:19.835 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-18T00:59:18.912425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1204381921,"num_objects":1983,"num_object_clones":445,"num_object_copies":3966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1983,"num_whiteouts":859,"num_read":175754,"num_read_kb":803419,"num_write":15801,"num_write_kb":549699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2926,"num_bytes_recovered":207217829,"num_keys_recovered":402,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12707,"ondisk_log_size":12707,"up":82,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":640252,"kb_used_data":488396,"kb_used_omap":7,"kb_used_meta":151672,"kb_avail":471218948,"statfs":{"total":483183820800,"available":482528202752,"internally_reserved":0,"allocated":500117504,"data_stored":594118262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7969,"internal_metadata":155312351},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":134851965,"num_objects":59,"num_object_clones":22,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":21,"num_read":2776,"num_read_kb":22057,"num_write":491,"num_write_kb":5514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.860510"},"pg_stats":[{"pgid":"3.1f","version":"102'380","reported_seq":1881,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.062830+0000","last_change":"2024-09-18T00:57:38.909849+0000","last_active":"2024-09-18T00:59:15.062830+0000","last_peered":"2024-09-18T00:59:15.062830+0000","last_clean":"2024-09-18T00:59:15.062830+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T00:59:15.062830+0000","last_undegraded":"2024-09-18T00:59:15.062830+0000","last_fullsized":"2024-09-18T00:59:15.062830+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:24:20.402015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001565786,"stat_sum":{"num_bytes":12596951,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":3325,"num_read_kb":7172,"num_write":395,"num_write_kb":5838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"102'392","reported_seq":3675,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.066483+0000","last_change":"2024-09-18T00:59:04.996334+0000","last_active":"2024-09-18T00:59:15.066483+0000","last_peered":"2024-09-18T00:59:15.066483+0000","last_clean":"2024-09-18T00:59:15.066483+0000","last_became_active":"2024-09-18T00:59:04.993158+0000","last_became_peered":"2024-09-18T00:59:04.993158+0000","last_unstale":"2024-09-18T00:59:15.066483+0000","last_undegraded":"2024-09-18T00:59:15.066483+0000","last_fullsized":"2024-09-18T00:59:15.066483+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:36:12.828834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040574399999999998,"stat_sum":{"num_bytes":57557150,"num_objects":80,"num_object_clones":18,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":37,"num_read":3343,"num_read_kb":23862,"num_write":439,"num_write_kb":10359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":26102183,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"102'646","reported_seq":11240,"reported_epoch":104,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:15.058381+0000","last_change":"2024-09-18T00:59:15.058381+0000","last_active":"2024-09-18T00:59:14.046292+0000","last_peered":"2024-09-18T00:59:13.909876+0000","last_clean":"2024-09-18T00:59:13.909876+0000","last_became_active":"2024-09-18T00:59:04.983725+0000","last_became_peered":"2024-09-18T00:59:04.983725+0000","last_unstale":"2024-09-18T00:59:15.058381+0000","last_undegraded":"2024-09-18T00:59:15.058381+0000","last_fullsized":"2024-09-18T00:59:15.058381+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039089,"stat_sum":{"num_bytes":42674614,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":36,"num_read":12947,"num_read_kb":39767,"num_write":845,"num_write_kb":7542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":4309906,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"102'452","reported_seq":2968,"reported_epoch":103,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:14.046535+0000","last_change":"2024-09-18T00:59:15.058014+0000","last_active":"2024-09-18T00:59:14.046535+0000","last_peered":"2024-09-18T00:59:13.239387+0000","last_clean":"2024-09-18T00:59:13.239387+0000","last_became_active":"2024-09-18T00:57:36.906003+0000","last_became_peered":"2024-09-18T00:57:36.906003+0000","last_unstale":"2024-09-18T00:59:14.046535+0000","last_undegraded":"2024-09-18T00:59:14.046535+0000","last_fullsized":"2024-09-18T00:59:14.046535+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000711266,"stat_sum":{"num_bytes":18082490,"num_objects":67,"num_object_clones":15,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":29,"num_read":4059,"num_read_kb":22070,"num_write":518,"num_write_kb":6441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"102'602","reported_seq":8362,"reported_epoch":103,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:14.054048+0000","last_change":"2024-09-18T00:59:15.058896+0000","last_active":"2024-09-18T00:59:14.054048+0000","last_peered":"2024-09-18T00:59:14.040947+0000","last_clean":"2024-09-18T00:59:14.040947+0000","last_became_active":"2024-09-18T00:57:36.913393+0000","last_became_peered":"2024-09-18T00:57:36.913393+0000","last_unstale":"2024-09-18T00:59:14.054048+0000","last_undegraded":"2024-09-18T00:59:14.054048+0000","last_fullsized":"2024-09-18T00:59:14.054048+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002554022,"stat_sum":{"num_bytes":68087084,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":25,"num_read":10590,"num_read_kb":38519,"num_write":753,"num_write_kb":24367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"102'250","reported_seq":3627,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.058385+0000","last_change":"2024-09-18T00:57:38.901805+0000","last_active":"2024-09-18T00:59:15.058385+0000","last_peered":"2024-09-18T00:59:15.058385+0000","last_clean":"2024-09-18T00:59:15.058385+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T00:59:15.058385+0000","last_undegraded":"2024-09-18T00:59:15.058385+0000","last_fullsized":"2024-09-18T00:59:15.058385+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:21:29.215466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098070899999999996,"stat_sum":{"num_bytes":40793974,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":4181,"num_read_kb":26900,"num_write":243,"num_write_kb":9041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"100'339","reported_seq":29508,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.055017+0000","last_change":"2024-09-18T00:57:38.922010+0000","last_active":"2024-09-18T00:59:15.055017+0000","last_peered":"2024-09-18T00:59:15.055017+0000","last_clean":"2024-09-18T00:59:15.055017+0000","last_became_active":"2024-09-18T00:57:36.910457+0000","last_became_peered":"2024-09-18T00:57:36.910457+0000","last_unstale":"2024-09-18T00:59:15.055017+0000","last_undegraded":"2024-09-18T00:59:15.055017+0000","last_fullsized":"2024-09-18T00:59:15.055017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:01:39.868673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026842849999999998,"stat_sum":{"num_bytes":79557268,"num_objects":68,"num_object_clones":15,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":29,"num_read":29099,"num_read_kb":76617,"num_write":339,"num_write_kb":22085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"102'521","reported_seq":3247,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.062211+0000","last_change":"2024-09-18T00:57:38.908144+0000","last_active":"2024-09-18T00:59:15.062211+0000","last_peered":"2024-09-18T00:59:15.062211+0000","last_clean":"2024-09-18T00:59:15.062211+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T00:59:15.062211+0000","last_undegraded":"2024-09-18T00:59:15.062211+0000","last_fullsized":"2024-09-18T00:59:15.062211+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:38:25.568815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000871597,"stat_sum":{"num_bytes":50095857,"num_objects":60,"num_object_clones":11,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":28,"num_read":4412,"num_read_kb":8340,"num_write":735,"num_write_kb":36722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"100'516","reported_seq":6334,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.063300+0000","last_change":"2024-09-18T00:59:04.992114+0000","last_active":"2024-09-18T00:59:15.063300+0000","last_peered":"2024-09-18T00:59:15.063300+0000","last_clean":"2024-09-18T00:59:15.063300+0000","last_became_active":"2024-09-18T00:59:04.989077+0000","last_became_peered":"2024-09-18T00:59:04.989077+0000","last_unstale":"2024-09-18T00:59:15.063300+0000","last_undegraded":"2024-09-18T00:59:15.063300+0000","last_fullsized":"2024-09-18T00:59:15.063300+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:39:33.623547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001269769,"stat_sum":{"num_bytes":47029733,"num_objects":65,"num_object_clones":14,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":21,"num_read":6945,"num_read_kb":45246,"num_write":646,"num_write_kb":24232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":7794778,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"103'345","reported_seq":5353,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.055623+0000","last_change":"2024-09-18T00:59:04.994621+0000","last_active":"2024-09-18T00:59:15.055623+0000","last_peered":"2024-09-18T00:59:15.055623+0000","last_clean":"2024-09-18T00:59:15.055623+0000","last_became_active":"2024-09-18T00:59:04.991534+0000","last_became_peered":"2024-09-18T00:59:04.991534+0000","last_unstale":"2024-09-18T00:59:15.055623+0000","last_undegraded":"2024-09-18T00:59:15.055623+0000","last_fullsized":"2024-09-18T00:59:15.055623+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:11.366691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076552,"stat_sum":{"num_bytes":32965955,"num_objects":69,"num_object_clones":20,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":26,"num_read":5087,"num_read_kb":21216,"num_write":481,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":16561805,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"99'387","reported_seq":1811,"reported_epoch":104,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:15.065650+0000","last_change":"2024-09-18T00:59:15.065650+0000","last_active":"2024-09-18T00:59:14.061439+0000","last_peered":"2024-09-18T00:59:09.135373+0000","last_clean":"2024-09-18T00:59:09.135373+0000","last_became_active":"2024-09-18T00:57:36.902977+0000","last_became_peered":"2024-09-18T00:57:36.902977+0000","last_unstale":"2024-09-18T00:59:15.065650+0000","last_undegraded":"2024-09-18T00:59:15.065650+0000","last_fullsized":"2024-09-18T00:59:15.065650+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051351900000000004,"stat_sum":{"num_bytes":41079127,"num_objects":59,"num_object_clones":10,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":28,"num_read":2048,"num_read_kb":14346,"num_write":692,"num_write_kb":45066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"102'193","reported_seq":1495,"reported_epoch":104,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:15.056516+0000","last_change":"2024-09-18T00:59:15.056516+0000","last_active":"2024-09-18T00:59:14.054565+0000","last_peered":"2024-09-18T00:59:13.043002+0000","last_clean":"2024-09-18T00:59:13.043002+0000","last_became_active":"2024-09-18T00:59:04.992342+0000","last_became_peered":"2024-09-18T00:59:04.992342+0000","last_unstale":"2024-09-18T00:59:15.056516+0000","last_undegraded":"2024-09-18T00:59:15.056516+0000","last_fullsized":"2024-09-18T00:59:15.056516+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028223319999999999,"stat_sum":{"num_bytes":7769886,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":24,"num_read":890,"num_read_kb":1195,"num_write":159,"num_write_kb":1215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":2312968,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.070930+0000","last_change":"2024-09-18T00:59:15.069224+0000","last_active":"2024-09-18T00:59:15.070930+0000","last_peered":"2024-09-18T00:59:15.070930+0000","last_clean":"2024-09-18T00:59:15.070930+0000","last_became_active":"2024-09-18T00:59:15.068326+0000","last_became_peered":"2024-09-18T00:59:15.068326+0000","last_unstale":"2024-09-18T00:59:15.070930+0000","last_undegraded":"2024-09-18T00:59:15.070930+0000","last_fullsized":"2024-09-18T00:59:15.070930+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:45.667754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"102'526","reported_seq":3713,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.055789+0000","last_change":"2024-09-18T00:57:38.920098+0000","last_active":"2024-09-18T00:59:15.055789+0000","last_peered":"2024-09-18T00:59:15.055789+0000","last_clean":"2024-09-18T00:59:15.055789+0000","last_became_active":"2024-09-18T00:57:36.920599+0000","last_became_peered":"2024-09-18T00:57:36.920599+0000","last_unstale":"2024-09-18T00:59:15.055789+0000","last_undegraded":"2024-09-18T00:59:15.055789+0000","last_fullsized":"2024-09-18T00:59:15.055789+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:50:46.161801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001276634,"stat_sum":{"num_bytes":27730768,"num_objects":57,"num_object_clones":9,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":4936,"num_read_kb":22021,"num_write":778,"num_write_kb":3969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":186,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.073941+0000","last_change":"2024-09-18T00:59:15.073941+0000","last_active":"2024-09-18T00:59:15.073941+0000","last_peered":"2024-09-18T00:59:15.073941+0000","last_clean":"2024-09-18T00:59:15.073941+0000","last_became_active":"2024-09-18T00:59:15.072986+0000","last_became_peered":"2024-09-18T00:59:15.072986+0000","last_unstale":"2024-09-18T00:59:15.073941+0000","last_undegraded":"2024-09-18T00:59:15.073941+0000","last_fullsized":"2024-09-18T00:59:15.073941+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:16.671308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"102'285","reported_seq":5884,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.055249+0000","last_change":"2024-09-18T00:57:38.920634+0000","last_active":"2024-09-18T00:59:15.055249+0000","last_peered":"2024-09-18T00:59:15.055249+0000","last_clean":"2024-09-18T00:59:15.055249+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T00:59:15.055249+0000","last_undegraded":"2024-09-18T00:59:15.055249+0000","last_fullsized":"2024-09-18T00:59:15.055249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:35.281910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728301,"stat_sum":{"num_bytes":30552470,"num_objects":63,"num_object_clones":18,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":28,"num_read":6219,"num_read_kb":13123,"num_write":233,"num_write_kb":10503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.055394+0000","last_change":"2024-09-18T00:59:01.917810+0000","last_active":"2024-09-18T00:59:15.055394+0000","last_peered":"2024-09-18T00:59:15.055394+0000","last_clean":"2024-09-18T00:59:15.055394+0000","last_became_active":"2024-09-18T00:59:01.916939+0000","last_became_peered":"2024-09-18T00:59:01.916939+0000","last_unstale":"2024-09-18T00:59:15.055394+0000","last_undegraded":"2024-09-18T00:59:15.055394+0000","last_fullsized":"2024-09-18T00:59:15.055394+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:19:05.860921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"102'216","reported_seq":3667,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.065968+0000","last_change":"2024-09-18T00:57:39.056519+0000","last_active":"2024-09-18T00:59:15.065968+0000","last_peered":"2024-09-18T00:59:15.065968+0000","last_clean":"2024-09-18T00:59:15.065968+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T00:59:15.065968+0000","last_undegraded":"2024-09-18T00:59:15.065968+0000","last_fullsized":"2024-09-18T00:59:15.065968+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:02:07.819554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043106399999999999,"stat_sum":{"num_bytes":41207980,"num_objects":58,"num_object_clones":13,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":27,"num_read":3353,"num_read_kb":26981,"num_write":197,"num_write_kb":16078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":180,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.062890+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T00:59:15.062890+0000","last_peered":"2024-09-18T00:59:15.062890+0000","last_clean":"2024-09-18T00:59:15.062890+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T00:59:15.062890+0000","last_undegraded":"2024-09-18T00:59:15.062890+0000","last_fullsized":"2024-09-18T00:59:15.062890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"102'412","reported_seq":3583,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.066072+0000","last_change":"2024-09-18T00:58:58.965665+0000","last_active":"2024-09-18T00:59:15.066072+0000","last_peered":"2024-09-18T00:59:15.066072+0000","last_clean":"2024-09-18T00:59:15.066072+0000","last_became_active":"2024-09-18T00:58:58.963883+0000","last_became_peered":"2024-09-18T00:58:58.963883+0000","last_unstale":"2024-09-18T00:59:15.066072+0000","last_undegraded":"2024-09-18T00:59:15.066072+0000","last_fullsized":"2024-09-18T00:59:15.066072+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:15:22.946139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000584944,"stat_sum":{"num_bytes":35187891,"num_objects":54,"num_object_clones":10,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":4209,"num_read_kb":27045,"num_write":670,"num_write_kb":12515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"102'447","reported_seq":3888,"reported_epoch":104,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:15.056361+0000","last_change":"2024-09-18T00:59:15.056361+0000","last_active":"2024-09-18T00:59:14.055892+0000","last_peered":"2024-09-18T00:59:13.187806+0000","last_clean":"2024-09-18T00:59:13.187806+0000","last_became_active":"2024-09-18T00:58:58.965751+0000","last_became_peered":"2024-09-18T00:58:58.965751+0000","last_unstale":"2024-09-18T00:59:15.056361+0000","last_undegraded":"2024-09-18T00:59:15.056361+0000","last_fullsized":"2024-09-18T00:59:15.056361+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056105599999999999,"stat_sum":{"num_bytes":60447742,"num_objects":57,"num_object_clones":12,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":4805,"num_read_kb":15367,"num_write":555,"num_write_kb":21372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":59707869,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":55,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.065049+0000","last_change":"2024-09-18T00:59:01.926313+0000","last_active":"2024-09-18T00:59:15.065049+0000","last_peered":"2024-09-18T00:59:15.065049+0000","last_clean":"2024-09-18T00:59:15.065049+0000","last_became_active":"2024-09-18T00:59:01.917338+0000","last_became_peered":"2024-09-18T00:59:01.917338+0000","last_unstale":"2024-09-18T00:59:15.065049+0000","last_undegraded":"2024-09-18T00:59:15.065049+0000","last_fullsized":"2024-09-18T00:59:15.065049+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:05:06.044579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"102'352","reported_seq":6645,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.061912+0000","last_change":"2024-09-18T00:57:38.907753+0000","last_active":"2024-09-18T00:59:15.061912+0000","last_peered":"2024-09-18T00:59:15.061912+0000","last_clean":"2024-09-18T00:59:15.061912+0000","last_became_active":"2024-09-18T00:57:36.907660+0000","last_became_peered":"2024-09-18T00:57:36.907660+0000","last_unstale":"2024-09-18T00:59:15.061912+0000","last_undegraded":"2024-09-18T00:59:15.061912+0000","last_fullsized":"2024-09-18T00:59:15.061912+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:03.417321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090282999999999997,"stat_sum":{"num_bytes":44809479,"num_objects":82,"num_object_clones":22,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":36,"num_read":6716,"num_read_kb":26335,"num_write":413,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":175,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-18T00:59:13.041560+0000","last_change":"2024-09-18T00:57:29.654283+0000","last_active":"2024-09-18T00:59:13.041560+0000","last_peered":"2024-09-18T00:59:13.041560+0000","last_clean":"2024-09-18T00:59:13.041560+0000","last_became_active":"2024-09-18T00:57:27.316780+0000","last_became_peered":"2024-09-18T00:57:27.316780+0000","last_unstale":"2024-09-18T00:59:13.041560+0000","last_undegraded":"2024-09-18T00:59:13.041560+0000","last_fullsized":"2024-09-18T00:59:13.041560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:54:03.363750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079685499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"102'345","reported_seq":6520,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.064818+0000","last_change":"2024-09-18T00:58:58.966926+0000","last_active":"2024-09-18T00:59:15.064818+0000","last_peered":"2024-09-18T00:59:15.064818+0000","last_clean":"2024-09-18T00:59:15.064818+0000","last_became_active":"2024-09-18T00:58:58.964827+0000","last_became_peered":"2024-09-18T00:58:58.964827+0000","last_unstale":"2024-09-18T00:59:15.064818+0000","last_undegraded":"2024-09-18T00:59:15.064818+0000","last_fullsized":"2024-09-18T00:59:15.064818+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:49.640026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045206800000000002,"stat_sum":{"num_bytes":36164222,"num_objects":71,"num_object_clones":15,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":27,"num_read":7012,"num_read_kb":35790,"num_write":363,"num_write_kb":16223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":9938529,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.062118+0000","last_change":"2024-09-18T00:59:01.918199+0000","last_active":"2024-09-18T00:59:15.062118+0000","last_peered":"2024-09-18T00:59:15.062118+0000","last_clean":"2024-09-18T00:59:15.062118+0000","last_became_active":"2024-09-18T00:59:01.916643+0000","last_became_peered":"2024-09-18T00:59:01.916643+0000","last_unstale":"2024-09-18T00:59:15.062118+0000","last_undegraded":"2024-09-18T00:59:15.062118+0000","last_fullsized":"2024-09-18T00:59:15.062118+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:22.282208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"101'562","reported_seq":3936,"reported_epoch":103,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:14.054018+0000","last_change":"2024-09-18T00:59:15.056371+0000","last_active":"2024-09-18T00:59:14.054018+0000","last_peered":"2024-09-18T00:59:13.048686+0000","last_clean":"2024-09-18T00:59:13.048686+0000","last_became_active":"2024-09-18T00:58:58.963566+0000","last_became_peered":"2024-09-18T00:58:58.963566+0000","last_unstale":"2024-09-18T00:59:14.054018+0000","last_undegraded":"2024-09-18T00:59:14.054018+0000","last_fullsized":"2024-09-18T00:59:14.054018+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064609,"stat_sum":{"num_bytes":24431489,"num_objects":60,"num_object_clones":8,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":30,"num_read":4527,"num_read_kb":13851,"num_write":977,"num_write_kb":39137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":8652757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":200,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.056977+0000","last_change":"2024-09-18T00:59:01.921709+0000","last_active":"2024-09-18T00:59:15.056977+0000","last_peered":"2024-09-18T00:59:15.056977+0000","last_clean":"2024-09-18T00:59:15.056977+0000","last_became_active":"2024-09-18T00:59:01.916229+0000","last_became_peered":"2024-09-18T00:59:01.916229+0000","last_unstale":"2024-09-18T00:59:15.056977+0000","last_undegraded":"2024-09-18T00:59:15.056977+0000","last_fullsized":"2024-09-18T00:59:15.056977+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:26.611604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":332,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.057099+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T00:59:15.057099+0000","last_peered":"2024-09-18T00:59:15.057099+0000","last_clean":"2024-09-18T00:59:15.057099+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T00:59:15.057099+0000","last_undegraded":"2024-09-18T00:59:15.057099+0000","last_fullsized":"2024-09-18T00:59:15.057099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"102'328","reported_seq":3646,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.054788+0000","last_change":"2024-09-18T00:57:38.919534+0000","last_active":"2024-09-18T00:59:15.054788+0000","last_peered":"2024-09-18T00:59:15.054788+0000","last_clean":"2024-09-18T00:59:15.054788+0000","last_became_active":"2024-09-18T00:57:36.908955+0000","last_became_peered":"2024-09-18T00:57:36.908955+0000","last_unstale":"2024-09-18T00:59:15.054788+0000","last_undegraded":"2024-09-18T00:59:15.054788+0000","last_fullsized":"2024-09-18T00:59:15.054788+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:27:19.413486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077760500000000001,"stat_sum":{"num_bytes":44547141,"num_objects":67,"num_object_clones":15,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":30,"num_read":4018,"num_read_kb":37968,"num_write":379,"num_write_kb":34965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"102'255","reported_seq":2474,"reported_epoch":103,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:14.054048+0000","last_change":"2024-09-18T00:59:15.056260+0000","last_active":"2024-09-18T00:59:14.054048+0000","last_peered":"2024-09-18T00:59:13.358847+0000","last_clean":"2024-09-18T00:59:13.358847+0000","last_became_active":"2024-09-18T00:59:04.993568+0000","last_became_peered":"2024-09-18T00:59:04.993568+0000","last_unstale":"2024-09-18T00:59:14.054048+0000","last_undegraded":"2024-09-18T00:59:14.054048+0000","last_fullsized":"2024-09-18T00:59:14.054048+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061570699999999995,"stat_sum":{"num_bytes":26034438,"num_objects":56,"num_object_clones":13,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":29,"num_read":2338,"num_read_kb":8115,"num_write":233,"num_write_kb":19570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":8365403,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"102'405","reported_seq":3044,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.057234+0000","last_change":"2024-09-18T00:59:04.993223+0000","last_active":"2024-09-18T00:59:15.057234+0000","last_peered":"2024-09-18T00:59:15.057234+0000","last_clean":"2024-09-18T00:59:15.057234+0000","last_became_active":"2024-09-18T00:59:04.992636+0000","last_became_peered":"2024-09-18T00:59:04.992636+0000","last_unstale":"2024-09-18T00:59:15.057234+0000","last_undegraded":"2024-09-18T00:59:15.057234+0000","last_fullsized":"2024-09-18T00:59:15.057234+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:42:34.778855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097047300000000002,"stat_sum":{"num_bytes":33290336,"num_objects":63,"num_object_clones":9,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":26,"num_read":3101,"num_read_kb":22976,"num_write":436,"num_write_kb":11443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":1719706,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"102'337","reported_seq":3298,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-18T00:59:13.358869+0000","last_change":"2024-09-18T00:57:38.899763+0000","last_active":"2024-09-18T00:59:13.358869+0000","last_peered":"2024-09-18T00:59:13.358869+0000","last_clean":"2024-09-18T00:59:13.358869+0000","last_became_active":"2024-09-18T00:57:36.905869+0000","last_became_peered":"2024-09-18T00:57:36.905869+0000","last_unstale":"2024-09-18T00:59:13.358869+0000","last_undegraded":"2024-09-18T00:59:13.358869+0000","last_fullsized":"2024-09-18T00:59:13.358869+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:16:24.596227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097615600000000005,"stat_sum":{"num_bytes":27974070,"num_objects":74,"num_object_clones":22,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":29,"num_read":3530,"num_read_kb":40812,"num_write":373,"num_write_kb":11128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"99'378","reported_seq":4692,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T00:59:10.081628+0000","last_change":"2024-09-18T00:57:38.909165+0000","last_active":"2024-09-18T00:59:10.081628+0000","last_peered":"2024-09-18T00:59:10.081628+0000","last_clean":"2024-09-18T00:59:10.081628+0000","last_became_active":"2024-09-18T00:57:36.908288+0000","last_became_peered":"2024-09-18T00:57:36.908288+0000","last_unstale":"2024-09-18T00:59:10.081628+0000","last_undegraded":"2024-09-18T00:59:10.081628+0000","last_fullsized":"2024-09-18T00:59:10.081628+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:20:36.557567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044375399999999998,"stat_sum":{"num_bytes":34475138,"num_objects":62,"num_object_clones":18,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":22,"num_read":5075,"num_read_kb":30514,"num_write":495,"num_write_kb":17132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"102'633","reported_seq":3836,"reported_epoch":103,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:14.047358+0000","last_change":"2024-09-18T00:59:15.058765+0000","last_active":"2024-09-18T00:59:14.047358+0000","last_peered":"2024-09-18T00:59:13.359887+0000","last_clean":"2024-09-18T00:59:13.359887+0000","last_became_active":"2024-09-18T00:59:04.983302+0000","last_became_peered":"2024-09-18T00:59:04.983302+0000","last_unstale":"2024-09-18T00:59:14.047358+0000","last_undegraded":"2024-09-18T00:59:14.047358+0000","last_fullsized":"2024-09-18T00:59:14.047358+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023201229999999999,"stat_sum":{"num_bytes":63630447,"num_objects":78,"num_object_clones":19,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":33,"num_read":6005,"num_read_kb":37907,"num_write":862,"num_write_kb":23033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13370804,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"99'524","reported_seq":2250,"reported_epoch":104,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:15.065931+0000","last_change":"2024-09-18T00:59:15.065931+0000","last_active":"2024-09-18T00:59:14.047261+0000","last_peered":"2024-09-18T00:59:09.102721+0000","last_clean":"2024-09-18T00:59:09.102721+0000","last_became_active":"2024-09-18T00:58:58.964808+0000","last_became_peered":"2024-09-18T00:58:58.964808+0000","last_unstale":"2024-09-18T00:59:15.065931+0000","last_undegraded":"2024-09-18T00:59:15.065931+0000","last_fullsized":"2024-09-18T00:59:15.065931+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090629000000000003,"stat_sum":{"num_bytes":30344317,"num_objects":60,"num_object_clones":15,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":23,"num_read":3230,"num_read_kb":14561,"num_write":804,"num_write_kb":20616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":9652505,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"102'350","reported_seq":3278,"reported_epoch":104,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:15.058887+0000","last_change":"2024-09-18T00:59:15.058887+0000","last_active":"2024-09-18T00:59:14.046287+0000","last_peered":"2024-09-18T00:59:14.014208+0000","last_clean":"2024-09-18T00:59:14.014208+0000","last_became_active":"2024-09-18T00:57:36.905462+0000","last_became_peered":"2024-09-18T00:57:36.905462+0000","last_unstale":"2024-09-18T00:59:15.058887+0000","last_undegraded":"2024-09-18T00:59:15.058887+0000","last_fullsized":"2024-09-18T00:59:15.058887+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081831799999999995,"stat_sum":{"num_bytes":26138820,"num_objects":52,"num_object_clones":8,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":18,"num_read":4413,"num_read_kb":25149,"num_write":391,"num_write_kb":7920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"102'303","reported_seq":3863,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.057027+0000","last_change":"2024-09-18T00:57:38.900906+0000","last_active":"2024-09-18T00:59:15.057027+0000","last_peered":"2024-09-18T00:59:15.057027+0000","last_clean":"2024-09-18T00:59:15.057027+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T00:59:15.057027+0000","last_undegraded":"2024-09-18T00:59:15.057027+0000","last_fullsized":"2024-09-18T00:59:15.057027+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:49.375708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069386899999999995,"stat_sum":{"num_bytes":49871975,"num_objects":75,"num_object_clones":15,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":32,"num_read":4128,"num_read_kb":40533,"num_write":252,"num_write_kb":25794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"102'220","reported_seq":2563,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.057027+0000","last_change":"2024-09-18T00:59:04.992636+0000","last_active":"2024-09-18T00:59:15.057027+0000","last_peered":"2024-09-18T00:59:15.057027+0000","last_clean":"2024-09-18T00:59:15.057027+0000","last_became_active":"2024-09-18T00:59:04.991454+0000","last_became_peered":"2024-09-18T00:59:04.991454+0000","last_unstale":"2024-09-18T00:59:15.057027+0000","last_undegraded":"2024-09-18T00:59:15.057027+0000","last_fullsized":"2024-09-18T00:59:15.057027+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:08.066475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071487799999999995,"stat_sum":{"num_bytes":23790259,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":1797,"num_read_kb":12615,"num_write":181,"num_write_kb":13781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":13441939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"94'317","reported_seq":5380,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T00:59:15.064640+0000","last_change":"2024-09-18T00:59:04.995178+0000","last_active":"2024-09-18T00:59:15.064640+0000","last_peered":"2024-09-18T00:59:15.064640+0000","last_clean":"2024-09-18T00:59:15.064640+0000","last_became_active":"2024-09-18T00:59:04.993698+0000","last_became_peered":"2024-09-18T00:59:04.993698+0000","last_unstale":"2024-09-18T00:59:15.064640+0000","last_undegraded":"2024-09-18T00:59:15.064640+0000","last_fullsized":"2024-09-18T00:59:15.064640+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:03:23.060946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055104700000000004,"stat_sum":{"num_bytes":29967785,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":24,"num_read":5790,"num_read_kb":15518,"num_write":340,"num_write_kb":9756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":8283018,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"102'369","reported_seq":2610,"reported_epoch":104,"state":"remapped+peering","last_fresh":"2024-09-18T00:59:15.056364+0000","last_change":"2024-09-18T00:59:15.056364+0000","last_active":"2024-09-18T00:59:14.054293+0000","last_peered":"2024-09-18T00:59:13.043214+0000","last_clean":"2024-09-18T00:59:13.043214+0000","last_became_active":"2024-09-18T00:59:04.992349+0000","last_became_peered":"2024-09-18T00:59:04.992349+0000","last_unstale":"2024-09-18T00:59:15.056364+0000","last_undegraded":"2024-09-18T00:59:15.056364+0000","last_fullsized":"2024-09-18T00:59:15.056364+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070399799999999995,"stat_sum":{"num_bytes":14904678,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":19,"num_read":3510,"num_read_kb":10888,"num_write":463,"num_write_kb":6345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2320662,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1203791534,"num_objects":1979,"num_object_clones":445,"num_object_copies":3958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1979,"num_whiteouts":859,"num_read":175638,"num_read_kb":803319,"num_write":15640,"num_write_kb":547281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":207217810,"num_keys_recovered":402,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":746512384,"data_stored":905431478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12587,"ondisk_log_size":12587,"up":64,"acting":52,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":110728,"kb_used_data":83640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94261112,"statfs":{"total":96636764160,"available":96523378688,"internally_reserved":0,"allocated":85647360,"data_stored":101141120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509466,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135124,"kb_used_data":103932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94236716,"statfs":{"total":96636764160,"available":96498397184,"internally_reserved":0,"allocated":106426368,"data_stored":138735499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":164780,"kb_used_data":129432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94207060,"statfs":{"total":96636764160,"available":96468029440,"internally_reserved":0,"allocated":132538368,"data_stored":149982852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":64424509468,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":64424509468,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509467,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148416,"kb_used_data":117224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94223424,"statfs":{"total":96636764160,"available":96484786176,"internally_reserved":0,"allocated":120037376,"data_stored":141310338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":64424509467,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81204,"kb_used_data":54168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94290636,"statfs":{"total":96636764160,"available":96553611264,"internally_reserved":0,"allocated":55468032,"data_stored":62948453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136798208,"data_stored":170151383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":118210560,"data_stored":140151565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52674560,"data_stored":68201235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":63754240,"data_stored":76508402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":131309568,"data_stored":149414466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":104603648,"data_stored":137576745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":84414464,"data_stored":100572715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":54747136,"data_stored":62854967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:59:19.836 INFO:tasks.thrashosds.thrasher:Setting 3.18 to [7, 0, 5, 3] 2024-09-18T00:59:19.837 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.18', '7', '0', '5', '3'] 2024-09-18T00:59:19.837 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.18 7 0 5 3 2024-09-18T00:59:19.877 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:19.878 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:19.880 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:19.877+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:19.893 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:19.937 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:59:19.982 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:19.981+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:20.023 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:20.023 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:20.037 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:20.084 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:20.085+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:20.164 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:20.164 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:20.178 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:20.216 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone34 from 0x4cd8500 (overlap 0x30494e) to 0x0 2024-09-18T00:59:20.222 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:20.221+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:20.282 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:20.281+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:20.333 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-18T00:59:20.354 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:20.354 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:20.371 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:20.375 INFO:teuthology.orchestra.run.smithi042.stdout:4599 write 0x6f76daf thru 0x6f8053b (0x978d bytes) 2024-09-18T00:59:20.383 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:20.381+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:20.403 INFO:teuthology.orchestra.run.smithi042.stdout:4600 trunc from 0x6f8053c to 0xdb25316 2024-09-18T00:59:20.440 INFO:teuthology.orchestra.run.smithi042.stdout:4601 trunc from 0xdb25316 to 0x3ffbac0 2024-09-18T00:59:20.471 INFO:teuthology.orchestra.run.smithi042.stdout:4602 clone 36 order 23 su 524288 sc 11 2024-09-18T00:59:20.484 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:20.481+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:20.543 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:20.543 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:20.557 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:20.585 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:20.581+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:20.687 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:20.685+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:20.720 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:20.720 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:20.735 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:20.787 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:20.785+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:20.857 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:20.857 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:20.871 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:20.920 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:20.921+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:20.989 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:20.989+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:20.999 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:20.999 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:21.013 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:21.021 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:21.100 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:21.101+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:set 3.18 pg_upmap_items mapping to [7->0,5->3] 2024-09-18T00:59:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:59:21.183 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:21.183 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:21.189 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:21.185+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:21.192 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:21.193 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:21.200 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:21.200 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:21.208 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:21.217 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:21.217 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:21.219 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone35 intact 2024-09-18T00:59:21.233 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:21.234 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:21.251 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:21.251 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:21.271 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:21.271 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:21.290 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:21.290 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:21.292 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:21.293+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:21.311 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:21.344 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-18T00:59:21.377 INFO:teuthology.orchestra.run.smithi042.stdout:4603 write 0xa68c5d1 thru 0xa696237 (0x9c67 bytes) 2024-09-18T00:59:21.384 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 92.295377606000002, 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:21.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T00:59:21.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.389 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 91.904482654999995, 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T00:59:21.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T00:59:21.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T00:59:21.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T00:59:21.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78518 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1183744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 36.552951825999997, 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054532499999999998, 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78518 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831039+0000", 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831040+0000", 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3099999999999997e-07 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831042+0000", 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2879999999999999e-06 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831050+0000", 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.464e-06 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831055+0000", 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0819999999999998e-06 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885354+0000", 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054298739999999998 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885428+0000", 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4567999999999995e-05 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885572+0000", 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014372699999999999 2024-09-18T00:59:21.398 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78519 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1204224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 36.552929976999998, 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054707261, 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.399 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78519 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831061+0000", 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831062+0000", 2024-09-18T00:59:21.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:59:21.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831064+0000", 2024-09-18T00:59:21.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0360000000000001e-06 2024-09-18T00:59:21.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831069+0000", 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4149999999999998e-06 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831075+0000", 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2249999999999997e-06 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885602+0000", 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054526893 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885673+0000", 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0865000000000007e-05 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885768+0000", 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5422999999999998e-05 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.403 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78520 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1224704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 36.552909520999997, 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054906695999999998, 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78520 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.404 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831082+0000", 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0400000000000002e-07 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831084+0000", 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2960000000000001e-06 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831091+0000", 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:59:21.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831095+0000", 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.792e-06 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885802+0000", 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054706795000000003 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885884+0000", 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2297000000000002e-05 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.885988+0000", 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010393199999999999 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78521 3.12 3:4ee697aa:::rbd_data.117c304cc916.0000000000000004:1e [read 1245184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:21.408 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 36.552864648000003, 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055066616999999998, 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78521 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831126+0000", 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831127+0000", 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.99e-07 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831130+0000", 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6819999999999999e-06 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831137+0000", 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3849999999999996e-06 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831145+0000", 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7360000000000001e-06 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886020+0000", 2024-09-18T00:59:21.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054874700999999998 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886096+0000", 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6234000000000002e-05 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886193+0000", 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7280000000000004e-05 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78522 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 36.552838887, 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055137938999999997, 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78522 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831152+0000", 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831153+0000", 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.37e-07 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831155+0000", 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4420000000000001e-06 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831161+0000", 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831166+0000", 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8749999999999999e-06 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886279+0000", 2024-09-18T00:59:21.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055112337999999997 2024-09-18T00:59:21.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886290+0000", 2024-09-18T00:59:21.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1527e-05 2024-09-18T00:59:21.416 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.417 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.417 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.417 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78523 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:21.417 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:21.417 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 36.552818207000001, 2024-09-18T00:59:21.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055171425000000003, 2024-09-18T00:59:21.417 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.417 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78523 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831173+0000", 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0900000000000002e-07 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831176+0000", 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4830000000000002e-06 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831183+0000", 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0679999999999999e-06 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831189+0000", 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886331+0000", 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055142669999999998 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886344+0000", 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2918999999999999e-05 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78524 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 36.552796090999998, 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055194430000000003, 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.421 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78524 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831195+0000", 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831196+0000", 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1200000000000003e-07 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831198+0000", 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831204+0000", 2024-09-18T00:59:21.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2740000000000004e-06 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831209+0000", 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0900000000000004e-06 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886381+0000", 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055172142 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886389+0000", 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2150000000000001e-06 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:78525 3.12 3.55e96772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 36.552775578000002, 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055206535000000001, 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.425 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 78525 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.831216+0000", 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832358+0000", 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001142531 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832364+0000", 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832375+0000", 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1381e-05 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.832382+0000", 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9800000000000001e-06 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886415+0000", 2024-09-18T00:59:21.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054032678000000001 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:44.886422+0000", 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4449999999999999e-06 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108130 3.1 3:83363b4a:::rbd_data.117c8e0d79e4.000000000000002d:20 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:21.429 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.190060507999998, 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054351575999999999, 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108130 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:21.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193931+0000", 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1100000000000001e-07 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193933+0000", 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3690000000000001e-06 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193940+0000", 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1460000000000001e-06 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.193944+0000", 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9230000000000001e-06 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.245560+0000", 2024-09-18T00:59:21.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051615019999999998 2024-09-18T00:59:21.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.245633+0000", 2024-09-18T00:59:21.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3490000000000003e-05 2024-09-18T00:59:21.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.248282+0000", 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026492170000000002 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108393 3.1 3:86b5547c:::rbd_data.117c8e0d79e4.0000000000000037:20 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.186968953000001, 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054339392, 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.434 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108393 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197022+0000", 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5499999999999999e-07 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197024+0000", 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.308e-06 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197029+0000", 2024-09-18T00:59:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2024-09-18T00:59:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197032+0000", 2024-09-18T00:59:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.994e-06 2024-09-18T00:59:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251285+0000", 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054253006999999999 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251322+0000", 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7104000000000003e-05 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251362+0000", 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9941000000000003e-05 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108394 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.186906547, 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054381878000000002, 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108394 2024-09-18T00:59:21.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197085+0000", 2024-09-18T00:59:21.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5400000000000001e-07 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197088+0000", 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8830000000000002e-06 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197096+0000", 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8560000000000007e-06 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197101+0000", 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251388+0000", 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054287588999999997 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251427+0000", 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8665999999999999e-05 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251466+0000", 2024-09-18T00:59:21.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9369000000000002e-05 2024-09-18T00:59:21.443 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.443 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.443 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.443 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108395 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:21.443 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.186881132, 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054457976999999998, 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108395 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:21.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197110+0000", 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197111+0000", 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6000000000000004e-07 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197113+0000", 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4119999999999999e-06 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197119+0000", 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4160000000000001e-06 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197125+0000", 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8320000000000002e-06 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251492+0000", 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054366997 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251529+0000", 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6563000000000002e-05 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251568+0000", 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9196999999999998e-05 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.447 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108396 3.1 3:847b19e4:::rbd_data.117c8e0d79e4.000000000000001c:20 [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.186858060999999, 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054537624, 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108396 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.448 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197133+0000", 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6199999999999999e-07 2024-09-18T00:59:21.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197135+0000", 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197139+0000", 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1749999999999998e-06 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197143+0000", 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1350000000000001e-06 2024-09-18T00:59:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251594+0000", 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054451160999999998 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251632+0000", 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7840999999999999e-05 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251671+0000", 2024-09-18T00:59:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9169999999999999e-05 2024-09-18T00:59:21.452 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.452 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.452 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.452 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108413 3.1 3:8453e3d7:::rbd_data.117c8e0d79e4.0000000000000022:20 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:21.452 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:21.452 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.186414053, 2024-09-18T00:59:21.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054401959, 2024-09-18T00:59:21.452 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.452 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108413 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197577+0000", 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197578+0000", 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7199999999999999e-07 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197581+0000", 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8439999999999998e-06 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197587+0000", 2024-09-18T00:59:21.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1759999999999998e-06 2024-09-18T00:59:21.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197592+0000", 2024-09-18T00:59:21.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4310000000000002e-06 2024-09-18T00:59:21.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251903+0000", 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054310418999999999 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251939+0000", 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6310999999999998e-05 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.251979+0000", 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0206000000000001e-05 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.456 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.457 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.457 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108428 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:21.457 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:21.457 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.186219614999999, 2024-09-18T00:59:21.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054482228000000001, 2024-09-18T00:59:21.457 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.457 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.457 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.457 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108428 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:21.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197772+0000", 2024-09-18T00:59:21.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9900000000000001e-07 2024-09-18T00:59:21.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197774+0000", 2024-09-18T00:59:21.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.373e-06 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197781+0000", 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3130000000000004e-06 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197786+0000", 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9980000000000003e-06 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252117+0000", 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054331329999999997 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252178+0000", 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0952000000000002e-05 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252254+0000", 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5863000000000002e-05 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.461 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108429 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.186198665999999, 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054624675999999997, 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108429 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.462 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197792+0000", 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197793+0000", 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0600000000000001e-07 2024-09-18T00:59:21.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197795+0000", 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.458e-06 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197802+0000", 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.691e-06 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197807+0000", 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2249999999999999e-06 2024-09-18T00:59:21.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252283+0000", 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054476138 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252348+0000", 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4783999999999995e-05 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252417+0000", 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8973999999999994e-05 2024-09-18T00:59:21.465 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108430 3.1 3:836280f8:::rbd_data.117c8e0d79e4.0000000000000027:20 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.186176475, 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054759302000000003, 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.466 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108430 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197815+0000", 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.58e-07 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197818+0000", 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5600000000000001e-06 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197824+0000", 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.511e-06 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.197829+0000", 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1619999999999999e-06 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252446+0000", 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.054616988999999998 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252513+0000", 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6558999999999998e-05 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.252574+0000", 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1062999999999995e-05 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:108540 3.1 3:83363b4a:::rbd_data.117c8e0d79e4.000000000000002d:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.178797096, 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056511851000000002, 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.470 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.471 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.471 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.471 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.471 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 108540 2024-09-18T00:59:21.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.471 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:21.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205194+0000", 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.01e-07 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205197+0000", 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:59:21.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205204+0000", 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4610000000000002e-06 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.205210+0000", 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5140000000000001e-06 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.256876+0000", 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051666493000000001 2024-09-18T00:59:21.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.256915+0000", 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8724999999999999e-05 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:03.261706+0000", 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004791086 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.474 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:21.476 INFO:teuthology.orchestra.run.smithi042.stdout:4604 trunc from 0xa696238 to 0x1cd05b8 2024-09-18T00:59:21.476 INFO:teuthology.orchestra.run.smithi042.stdout:4605 read 0x1f21a8 thru 0x2020ec (0xff45 bytes) 2024-09-18T00:59:21.476 INFO:teuthology.orchestra.run.smithi042.stdout:4606 read 0x9ce532 thru 0x9d1f31 (0x3a00 bytes) 2024-09-18T00:59:21.476 INFO:teuthology.orchestra.run.smithi042.stdout:4608 read 0x14395 thru 0x17c41 (0x38ad bytes) 2024-09-18T00:59:21.476 INFO:teuthology.orchestra.run.smithi042.stdout:4610 read 0x14900ed thru 0x149d290 (0xd1a4 bytes) 2024-09-18T00:59:21.476 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:21.477 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:21.477 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:21.473+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:21.482 INFO:teuthology.orchestra.run.smithi042.stdout:4613 write 0xde3dc01 thru 0xde47644 (0x9a44 bytes) 2024-09-18T00:59:21.488 INFO:teuthology.orchestra.run.smithi042.stdout:4614 writesame 0xaa8deed thru 0xaa900bb data_size 0x5(0x21cf bytes) 2024-09-18T00:59:21.495 INFO:teuthology.orchestra.run.smithi042.stdout:4615 punch from 0xdbd176f to 0xdbd8509, (0x6d9a bytes) 2024-09-18T00:59:21.496 INFO:teuthology.orchestra.run.smithi042.stdout:4616 punch from 0x2885b99 to 0x2894516, (0xe97d bytes) 2024-09-18T00:59:21.496 INFO:teuthology.orchestra.run.smithi042.stdout:4618 punch from 0x95079f4 to 0x950bbfd, (0x4209 bytes) 2024-09-18T00:59:21.508 INFO:teuthology.orchestra.run.smithi042.stdout:4619 writesame 0xe453ff0 thru 0xe45d2e0 data_size 0x1(0x92f1 bytes) 2024-09-18T00:59:21.509 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:21.509 INFO:teuthology.orchestra.run.smithi042.stdout:4622 punch from 0x3ca256f to 0x3cadc3f, (0xb6d0 bytes) 2024-09-18T00:59:21.509 INFO:teuthology.orchestra.run.smithi042.stdout:4623 punch from 0xd5ff71d to 0xd60a1eb, (0xaace bytes) 2024-09-18T00:59:21.521 INFO:teuthology.orchestra.run.smithi042.stdout:4624 trunc from 0xe45d2e1 to 0xa01f4f2 2024-09-18T00:59:21.545 INFO:teuthology.orchestra.run.smithi042.stdout:4627 writesame 0x989ac7 thru 0x98f450 data_size 0x49(0x598a bytes) 2024-09-18T00:59:21.547 INFO:teuthology.orchestra.run.smithi042.stdout:4628 read 0x654124e thru 0x654ed0f (0xdac2 bytes) 2024-09-18T00:59:21.547 INFO:teuthology.orchestra.run.smithi042.stdout:4629 read 0x90221a5 thru 0x903066f (0xe4cb bytes) 2024-09-18T00:59:21.548 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:21.548 INFO:teuthology.orchestra.run.smithi042.stdout:4631 punch from 0x8c392e6 to 0x8c3a2e7, (0x1001 bytes) 2024-09-18T00:59:21.548 INFO:teuthology.orchestra.run.smithi042.stdout:4632 write 0x41437bd thru 0x414465b (0xe9f bytes) 2024-09-18T00:59:21.550 INFO:teuthology.orchestra.run.smithi042.stdout:4633 read 0xb2d07c thru 0xb3cb9e (0xfb23 bytes) 2024-09-18T00:59:21.551 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:21.551 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:21.565 INFO:teuthology.orchestra.run.smithi042.stdout:4638 write 0xaf4d4b0 thru 0xaf585fd (0xb14e bytes) 2024-09-18T00:59:21.570 INFO:teuthology.orchestra.run.smithi042.stdout:4639 read 0x9755e35 thru 0x975dc77 (0x7e43 bytes) 2024-09-18T00:59:21.570 INFO:teuthology.orchestra.run.smithi042.stdout:4640 write 0x6089209 thru 0x6098125 (0xef1d bytes) 2024-09-18T00:59:21.577 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:21.573+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:21.603 INFO:teuthology.orchestra.run.smithi042.stdout:4643 trunc from 0xaf585fe to 0xeefea1 2024-09-18T00:59:21.659 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:21.659 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:21.659 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:21.659 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:21.664 INFO:teuthology.orchestra.run.smithi042.stdout:4644 write 0x6d6e8d4 thru 0x6d783b7 (0x9ae4 bytes) 2024-09-18T00:59:21.670 INFO:teuthology.orchestra.run.smithi042.stdout:4645 write 0x5633ff6 thru 0x563e0fb (0xa106 bytes) 2024-09-18T00:59:21.674 INFO:teuthology.orchestra.run.smithi042.stdout:4648 read 0x44ba7b9 thru 0x44c05e2 (0x5e2a bytes) 2024-09-18T00:59:21.676 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:21.677 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:21.677+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:21.688 INFO:teuthology.orchestra.run.smithi042.stdout:4649 trunc from 0x6d783b8 to 0xbe079d2 2024-09-18T00:59:21.717 INFO:teuthology.orchestra.run.smithi042.stdout:4650 trunc from 0xbe079d2 to 0x511a3cd 2024-09-18T00:59:21.740 INFO:teuthology.orchestra.run.smithi042.stdout:4652 trunc from 0x511a3cd to 0x41955cd 2024-09-18T00:59:21.755 INFO:teuthology.orchestra.run.smithi042.stdout:4653 read 0x1dd0bbf thru 0x1dda2f3 (0x9735 bytes) 2024-09-18T00:59:21.756 INFO:teuthology.orchestra.run.smithi042.stdout:4656 read 0x3883b31 thru 0x388ff90 (0xc460 bytes) 2024-09-18T00:59:21.757 INFO:teuthology.orchestra.run.smithi042.stdout:4657 writesame 0x255156c thru 0x255d9f0 data_size 0x1(0xc485 bytes) 2024-09-18T00:59:21.779 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:21.777+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:21.807 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:21.808 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:21.808 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:21.818 INFO:teuthology.orchestra.run.smithi042.stdout:4658 read 0x2685ef8 thru 0x2694329 (0xe432 bytes) 2024-09-18T00:59:21.820 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:21.823 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:21.857 INFO:teuthology.orchestra.run.smithi042.stdout:4660 write 0xe3a8a0b thru 0xe3b527b (0xc871 bytes) 2024-09-18T00:59:21.863 INFO:teuthology.orchestra.run.smithi042.stdout:4661 writesame 0x1ab67f9 thru 0x1abfa7c data_size 0x2(0x9284 bytes) 2024-09-18T00:59:21.878 INFO:teuthology.orchestra.run.smithi042.stdout:4662 punch from 0x39c0acc to 0x39c3785, (0x2cb9 bytes) 2024-09-18T00:59:21.878 INFO:teuthology.orchestra.run.smithi042.stdout:4663 read 0x7d58a1d thru 0x7d63389 (0xa96d bytes) 2024-09-18T00:59:21.878 INFO:teuthology.orchestra.run.smithi042.stdout:4664 read 0x63fa7d8 thru 0x63fc33e (0x1b67 bytes) 2024-09-18T00:59:21.880 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:21.877+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:21.892 INFO:teuthology.orchestra.run.smithi042.stdout:4666 trunc from 0xe3b527c to 0x9387c86 2024-09-18T00:59:21.915 INFO:teuthology.orchestra.run.smithi042.stdout:4671 read 0x39f388f thru 0x39fc143 (0x88b5 bytes) 2024-09-18T00:59:21.945 INFO:teuthology.orchestra.run.smithi042.stdout:4672 writesame 0xe74bdbc thru 0xe74c1d0 data_size 0xb(0x415 bytes) 2024-09-18T00:59:21.945 INFO:teuthology.orchestra.run.smithi042.stdout:4674 writesame 0x7b33d56 thru 0x7b351f0 data_size 0x5(0x149b bytes) 2024-09-18T00:59:21.953 INFO:teuthology.orchestra.run.smithi042.stdout:4675 read 0x168b5e7 thru 0x169a073 (0xea8d bytes) 2024-09-18T00:59:21.955 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:21.955 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:21.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:21.955 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:21.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.955 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:21.955 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:21.955 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 96.948494904, 2024-09-18T00:59:21.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:59:21.955 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.955 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:21.959 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 96.948436646999994, 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:59:21.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 96.948378207999994, 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:59:21.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:21.967 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 96.948316990999999, 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:21.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:59:21.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:59:21.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:59:21.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:59:21.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:59:21.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:59:21.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:59:21.972 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.972 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.972 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.973 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:21.973 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:21.973 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 96.948226575999996, 2024-09-18T00:59:21.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:59:21.973 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.973 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.974 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.974 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.974 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.974 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.974 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:59:21.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:59:21.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 96.948155643000007, 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.978 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 96.948096036999999, 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 96.948036666999997, 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.986 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:59:21.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:59:21.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 96.947974877999997, 2024-09-18T00:59:21.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:59:21.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:59:21.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 96.947842054999995, 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:21.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:59:21.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:59:21.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:59:21.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.392624577999996, 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:21.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T00:59:22.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.004 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.392124453999998, 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:22.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:22.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:22.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:22.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T00:59:22.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T00:59:22.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:22.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T00:59:22.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T00:59:22.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:22.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T00:59:22.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T00:59:22.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:22.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:22.009 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.392093946000003, 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T00:59:22.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.013 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.392069281000005, 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:22.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:22.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:22.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T00:59:22.017 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.392040749000003, 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:22.018 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.391707457999999, 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:22.022 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:22.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:22.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T00:59:22.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T00:59:22.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T00:59:22.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T00:59:22.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:22.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T00:59:22.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T00:59:22.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:22.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T00:59:22.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T00:59:22.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:22.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T00:59:22.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T00:59:22.028 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.028 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:22.029 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.029 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:22.029 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:22.029 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.391665606999993, 2024-09-18T00:59:22.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T00:59:22.029 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:22.029 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:22.029 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:22.029 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T00:59:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T00:59:22.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.390628110999998, 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T00:59:22.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T00:59:22.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T00:59:22.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.738340106000003, 2024-09-18T00:59:22.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T00:59:22.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:22.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:22.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T00:59:22.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:138250 3.c 3:32730bfd:::rbd_data.117c1c3044f5.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:21.758914+0000", 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.19264229799999999, 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060687680000000001, 2024-09-18T00:59:22.045 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:22.046 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:22.046 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:22.046 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:22.046 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:22.046 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.048 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 138250 2024-09-18T00:59:22.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.048 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:22.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:22.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T00:59:22.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758917+0000", 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5550000000000001e-06 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758925+0000", 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5900000000000008e-06 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758947+0000", 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1197000000000001e-05 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758957+0000", 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0137e-05 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759094+0000", 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013775300000000001 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759246+0000", 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015127900000000001 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.770839+0000", 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011593538 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819413+0000", 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048573660999999997 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819467+0000", 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3795999999999998e-05 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819529+0000", 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2675999999999994e-05 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819602+0000", 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2497999999999996e-05 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.052 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:22.053 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.053 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:22.053 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:22.053 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:22.055 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:4676 write 0x36041d thru 0x369068 (0x8c4c bytes) 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:4680 write 0x76fc838 thru 0x7705b04 (0x92cd bytes) 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:4681 writesame 0x5b4b0f6 thru 0x5b591ec data_size 0x51(0xe0f7 bytes) 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:4683 punch from 0x418bd75 to 0x4194472, (0x86fd bytes) 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:4687 read 0xe321f27 thru 0xe32dee4 (0xbfbe bytes) 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:4689 write 0x2e6d93a thru 0x2e732ed (0x59b4 bytes) 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:4692 write 0xcdec67 thru 0xce0e6e (0x2208 bytes) 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:4693 punch from 0x3eaa1ae to 0x3eb874e, (0xe5a0 bytes) 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:4694 read 0xe16b8d6 thru 0xe16c891 (0xfbc bytes) 2024-09-18T00:59:22.056 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:22.057 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:22.057 INFO:teuthology.orchestra.run.smithi042.stdout:4698 write 0xb6ce596 thru 0xb6d5568 (0x6fd3 bytes) 2024-09-18T00:59:22.057 INFO:teuthology.orchestra.run.smithi042.stdout:4699 trunc from 0xe74c1d1 to 0xf6e30d 2024-09-18T00:59:22.057 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:22.053+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:22.057 INFO:teuthology.orchestra.run.smithi042.stdout:4700 read 0x1763ac thru 0x17664f (0x2a4 bytes) 2024-09-18T00:59:22.058 INFO:teuthology.orchestra.run.smithi042.stdout:4701 read 0xe33546 thru 0xe3aa9d (0x7558 bytes) 2024-09-18T00:59:22.101 INFO:teuthology.orchestra.run.smithi042.stdout:4703 writesame 0xbf35011 thru 0xbf3fa90 data_size 0x2c(0xaa80 bytes) 2024-09-18T00:59:22.102 INFO:teuthology.orchestra.run.smithi042.stdout:4705 writesame 0x9a682b4 thru 0x9a6e142 data_size 0x1(0x5e8f bytes) 2024-09-18T00:59:22.142 INFO:teuthology.orchestra.run.smithi042.stdout:4706 read 0x4dee77d thru 0x4df6535 (0x7db9 bytes) 2024-09-18T00:59:22.143 INFO:teuthology.orchestra.run.smithi042.stdout:4707 read 0x39e5acb thru 0x39f52eb (0xf821 bytes) 2024-09-18T00:59:22.155 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:22.153+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:22.165 INFO:teuthology.orchestra.run.smithi042.stdout:4708 trunc from 0xbf3fa91 to 0x2d9abf0 2024-09-18T00:59:22.209 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:22.209 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:22.209 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:22.209 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:22.215 INFO:teuthology.orchestra.run.smithi042.stdout:4709 trunc from 0x2d9abf0 to 0xafd2fc5 2024-09-18T00:59:22.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:22.230 INFO:teuthology.orchestra.run.smithi042.stdout:4710 writesame 0x9ac9671 thru 0x9acdb95 data_size 0x1(0x4525 bytes) 2024-09-18T00:59:22.255 INFO:teuthology.orchestra.run.smithi042.stdout:4711 write 0x4d48c2e thru 0x4d4e3f4 (0x57c7 bytes) 2024-09-18T00:59:22.255 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:22.253+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:22.274 INFO:teuthology.orchestra.run.smithi042.stdout:4712 write 0xbc0c101 thru 0xbc11b6c (0x5a6c bytes) 2024-09-18T00:59:22.282 INFO:teuthology.orchestra.run.smithi042.stdout:4714 write 0x998d1de thru 0x998e0ac (0xecf bytes) 2024-09-18T00:59:22.289 INFO:teuthology.orchestra.run.smithi042.stdout:4715 read 0x10f577a thru 0x10fd26c (0x7af3 bytes) 2024-09-18T00:59:22.290 INFO:teuthology.orchestra.run.smithi042.stdout:4716 punch from 0xa1ec18c to 0xa1f68be, (0xa732 bytes) 2024-09-18T00:59:22.290 INFO:teuthology.orchestra.run.smithi042.stdout:4718 read 0xb8c66d5 thru 0xb8c9866 (0x3192 bytes) 2024-09-18T00:59:22.290 INFO:teuthology.orchestra.run.smithi042.stdout:4719 read 0x830a8a9 thru 0x831027c (0x59d4 bytes) 2024-09-18T00:59:22.290 INFO:teuthology.orchestra.run.smithi042.stdout:4720 read 0xa04d149 thru 0xa056d87 (0x9c3f bytes) 2024-09-18T00:59:22.291 INFO:teuthology.orchestra.run.smithi042.stdout:4722 read 0x2f56a74 thru 0x2f59284 (0x2811 bytes) 2024-09-18T00:59:22.291 INFO:teuthology.orchestra.run.smithi042.stdout:4723 read 0x5d9cd48 thru 0x5da2f2b (0x61e4 bytes) 2024-09-18T00:59:22.291 INFO:teuthology.orchestra.run.smithi042.stdout:4724 write 0x51e4193 thru 0x51e7ed5 (0x3d43 bytes) 2024-09-18T00:59:22.309 INFO:teuthology.orchestra.run.smithi042.stdout:4726 write 0xc5ceadb thru 0xc5cf865 (0xd8b bytes) 2024-09-18T00:59:22.316 INFO:teuthology.orchestra.run.smithi042.stdout:4728 read 0x6cf7ff5 thru 0x6d02c25 (0xac31 bytes) 2024-09-18T00:59:22.316 INFO:teuthology.orchestra.run.smithi042.stdout:4729 read 0x756bb54 thru 0x756c676 (0xb23 bytes) 2024-09-18T00:59:22.333 INFO:teuthology.orchestra.run.smithi042.stdout:4730 trunc from 0xc5cf866 to 0x4a2a3cd 2024-09-18T00:59:22.356 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:22.353+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:22.365 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:22.365 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:22.365 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:22.365 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:22.365 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:22.368 INFO:teuthology.orchestra.run.smithi042.stdout:4732 read 0x102df57 thru 0x10351ee (0x7298 bytes) 2024-09-18T00:59:22.369 INFO:teuthology.orchestra.run.smithi042.stdout:4733 punch from 0x1ffe570 to 0x20059a6, (0x7436 bytes) 2024-09-18T00:59:22.369 INFO:teuthology.orchestra.run.smithi042.stdout:4734 read 0x41335d8 thru 0x413fcc2 (0xc6eb bytes) 2024-09-18T00:59:22.382 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:22.395 INFO:teuthology.orchestra.run.smithi042.stdout:4735 writesame 0x93096cf thru 0x9315806 data_size 0x6c(0xc138 bytes) 2024-09-18T00:59:22.395 INFO:teuthology.orchestra.run.smithi042.stdout:4737 writesame 0x5a2704f thru 0x5a2fa37 data_size 0x17(0x89e9 bytes) 2024-09-18T00:59:22.404 INFO:teuthology.orchestra.run.smithi042.stdout:4738 trunc from 0x9315807 to 0x8ce8641 2024-09-18T00:59:22.419 INFO:teuthology.orchestra.run.smithi042.stdout:4741 read 0x832e089 thru 0x833124c (0x31c4 bytes) 2024-09-18T00:59:22.435 INFO:teuthology.orchestra.run.smithi042.stdout:4744 writesame 0x9923a5c thru 0x992c7fc data_size 0x1(0x8da1 bytes) 2024-09-18T00:59:22.435 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:22.435 INFO:teuthology.orchestra.run.smithi042.stdout:4746 write 0x6e85fa6 thru 0x6e8f6d7 (0x9732 bytes) 2024-09-18T00:59:22.444 INFO:teuthology.orchestra.run.smithi042.stdout:4747 read 0x27288a2 thru 0x272bea6 (0x3605 bytes) 2024-09-18T00:59:22.445 INFO:teuthology.orchestra.run.smithi042.stdout:4749 read 0x7e6b983 thru 0x7e7aa35 (0xf0b3 bytes) 2024-09-18T00:59:22.445 INFO:teuthology.orchestra.run.smithi042.stdout:4750 read 0xf0b990 thru 0xf14405 (0x8a76 bytes) 2024-09-18T00:59:22.446 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:22.446 INFO:teuthology.orchestra.run.smithi042.stdout:4753 clone 37 order 19 su 262144 sc 15 2024-09-18T00:59:22.456 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:22.457+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:22.514 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:22.514 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.448622181000005, 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:59:22.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:59:22.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:59:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:59:22.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:22.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:59:22.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:59:22.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:22.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:59:22.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:59:22.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:22.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:59:22.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:59:22.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:59:22.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.556088362000001, 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T00:59:22.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T00:59:22.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.525 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.807021572, 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T00:59:22.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T00:59:22.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T00:59:22.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T00:59:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T00:59:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T00:59:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T00:59:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T00:59:22.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.796647116000001, 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T00:59:22.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.796574434, 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.535 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T00:59:22.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T00:59:22.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.796543568000001, 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T00:59:22.539 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.796490794, 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T00:59:22.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.796464628000001, 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T00:59:22.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.796439473, 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.796016162000001, 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.795977784, 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T00:59:22.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T00:59:22.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.795953019000001, 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T00:59:22.564 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T00:59:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T00:59:22.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.568 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.789158255, 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:22.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T00:59:22.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T00:59:22.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.572 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.789132655, 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.789109312000001, 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.789086873, 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:22.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T00:59:22.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.789066099999999, 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T00:59:22.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T00:59:22.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.789045074000001, 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T00:59:22.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T00:59:22.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.789004492, 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T00:59:22.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.788970393, 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:22.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:22.601 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:22.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:22.603 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:22.601+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:22.704 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:22.701+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:22.775 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:22.776 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:22.776 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:22.776 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:22.790 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:22.838 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:22.837+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:22.905 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:22.905+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:22.909 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:22.909 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:22.909 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:22.910 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:22.910 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:22.924 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:23.006 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:23.005+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:23.091 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.092 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 119.79996196, 2024-09-18T00:59:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:59:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:59:23.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:59:23.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:23.101 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.603620907000007, 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:23.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:59:23.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:59:23.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.603599470000006, 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:59:23.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:59:23.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:23.110 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.603504142999995, 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:59:23.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.603478703999997, 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.603456180999999, 2024-09-18T00:59:23.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:59:23.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.124 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.603432741000006, 2024-09-18T00:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:59:23.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:59:23.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.603408931000004, 2024-09-18T00:59:23.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:23.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:59:23.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.603386565999998, 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.134 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:59:23.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:59:23.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:23.138 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.603359948999994, 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:23.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.599208177999998, 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:59:23.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:59:23.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.599187435000005, 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.599141869999997, 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.156 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.599115529000002, 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.599092858999995, 2024-09-18T00:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:59:23.161 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.599074486000006, 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:59:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:59:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.599054698000003, 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.599027828000004, 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:59:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.522837359, 2024-09-18T00:59:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T00:59:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.1117722629999998, 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T00:59:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T00:59:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T00:59:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T00:59:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T00:59:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:23.190 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:23.190 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:59:23.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:23.193 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:23.189+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:23.292 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:23.289+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:23.330 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:23.331 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone36 from 0x992c7fd (overlap 0xeefea1) to 0x17e643 2024-09-18T00:59:23.379 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:23.379 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:23.379 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:23.379 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:23.393 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:23.393+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:23.395 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:23.479 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:23.479 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:23.494 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:23.493+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:23.497 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:23.497 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:23.517 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:23.517 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:23.535 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:23.536 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:23.566 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:23.573 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:23.573 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:23.577 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-18T00:59:23.582 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:23.606 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:23.607 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:23.607 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:23.605+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:23.629 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:23.651 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:23.651 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:23.709 INFO:teuthology.orchestra.run.smithi042.stdout:4754 trunc from 0x992c7fd to 0x8f3e763 2024-09-18T00:59:23.709 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:23.705+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:23.739 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:23.739 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:23.757 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:23.768 INFO:teuthology.orchestra.run.smithi042.stdout:4755 write 0xdddc3e1 thru 0xdde4ecf (0x8aef bytes) 2024-09-18T00:59:23.775 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:23.775 INFO:teuthology.orchestra.run.smithi042.stdout:4758 read 0x8b704dc thru 0x8b7b145 (0xac6a bytes) 2024-09-18T00:59:23.778 INFO:teuthology.orchestra.run.smithi042.stdout:4759 trunc from 0xdde4ed0 to 0xc3ef035 2024-09-18T00:59:23.796 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:23.797+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:23.807 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:23.807 INFO:teuthology.orchestra.run.smithi042.stdout:4761 read 0x58e719b thru 0x58e8bdc (0x1a42 bytes) 2024-09-18T00:59:23.808 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:23.808 INFO:teuthology.orchestra.run.smithi042.stdout:4763 punch from 0xbaee2cf to 0xbaf48b2, (0x65e3 bytes) 2024-09-18T00:59:23.809 INFO:teuthology.orchestra.run.smithi042.stdout:4764 trunc from 0xc3ef035 to 0xcc3d0e7 2024-09-18T00:59:23.829 INFO:teuthology.orchestra.run.smithi042.stdout:4765 punch from 0xf8bea7 to 0xf9a119, (0xe272 bytes) 2024-09-18T00:59:23.829 INFO:teuthology.orchestra.run.smithi042.stdout:4766 punch from 0x2a4bd2b to 0x2a58062, (0xc337 bytes) 2024-09-18T00:59:23.829 INFO:teuthology.orchestra.run.smithi042.stdout:4767 writesame 0x705a43b thru 0x706133a data_size 0x80(0x6f00 bytes) 2024-09-18T00:59:23.855 INFO:teuthology.orchestra.run.smithi042.stdout:4768 trunc from 0xcc3d0e7 to 0x19cf706 2024-09-18T00:59:23.894 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:23.894 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:23.897 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:23.897+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:23.909 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:23.950 INFO:teuthology.orchestra.run.smithi042.stdout:4769 write 0x856407d thru 0x85703b3 (0xc337 bytes) 2024-09-18T00:59:23.960 INFO:teuthology.orchestra.run.smithi042.stdout:4770 trunc from 0x85703b4 to 0x77525f9 2024-09-18T00:59:23.998 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:23.997+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:24.006 INFO:teuthology.orchestra.run.smithi042.stdout:4771 write 0xc57ee43 thru 0xc589bad (0xad6b bytes) 2024-09-18T00:59:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:4773 punch from 0x54494e9 to 0x5453b79, (0xa690 bytes) 2024-09-18T00:59:24.031 INFO:teuthology.orchestra.run.smithi042.stdout:4774 write 0xcf7f448 thru 0xcf84c61 (0x581a bytes) 2024-09-18T00:59:24.040 INFO:teuthology.orchestra.run.smithi042.stdout:4776 writesame 0xa7e2887 thru 0xa7f10c5 data_size 0x73(0xe83f bytes) 2024-09-18T00:59:24.049 INFO:teuthology.orchestra.run.smithi042.stdout:4777 read 0x161780f thru 0x161a759 (0x2f4b bytes) 2024-09-18T00:59:24.050 INFO:teuthology.orchestra.run.smithi042.stdout:4779 writesame 0x51b29b7 thru 0x51b7787 data_size 0x1(0x4dd1 bytes) 2024-09-18T00:59:24.080 INFO:teuthology.orchestra.run.smithi042.stdout:4781 write 0x13d3000 thru 0x13d302d (0x2e bytes) 2024-09-18T00:59:24.081 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:24.082 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:24.098 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:24.099 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:24.097+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:24.107 INFO:teuthology.orchestra.run.smithi042.stdout:4782 writesame 0xe9abef0 thru 0xe9b064e data_size 0xb(0x475f bytes) 2024-09-18T00:59:24.107 INFO:teuthology.orchestra.run.smithi042.stdout:4783 writesame 0xdbf1f75 thru 0xdbf39a4 data_size 0x8(0x1a30 bytes) 2024-09-18T00:59:24.115 INFO:teuthology.orchestra.run.smithi042.stdout:4784 write 0xd18571b thru 0xd191e93 (0xc779 bytes) 2024-09-18T00:59:24.122 INFO:teuthology.orchestra.run.smithi042.stdout:4785 read 0xb6a9690 thru 0xb6b39cd (0xa33e bytes) 2024-09-18T00:59:24.128 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:59:24.143 INFO:teuthology.orchestra.run.smithi042.stdout:4786 trunc from 0xe9b064f to 0x609efbe 2024-09-18T00:59:24.176 INFO:teuthology.orchestra.run.smithi042.stdout:4787 write 0x4d6cf9c thru 0x4d75212 (0x8277 bytes) 2024-09-18T00:59:24.196 INFO:teuthology.orchestra.run.smithi042.stdout:4788 writesame 0x60a9f1b thru 0x60b9e73 data_size 0x1(0xff59 bytes) 2024-09-18T00:59:24.200 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:24.197+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:24.216 INFO:teuthology.orchestra.run.smithi042.stdout:4789 write 0x8b0129c thru 0x8b10fc1 (0xfd26 bytes) 2024-09-18T00:59:24.224 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xee4b677 2024-09-18T00:59:24.238 INFO:teuthology.orchestra.run.smithi042.stdout:4790 trunc from 0x8b10fc2 to 0xee4b677 2024-09-18T00:59:24.244 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:24.244 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:24.252 INFO:teuthology.orchestra.run.smithi042.stdout:4791 read 0x9449816 thru 0x94532be (0x9aa9 bytes) 2024-09-18T00:59:24.252 INFO:teuthology.orchestra.run.smithi042.stdout:4792 read 0xa92f19c thru 0xa935c0b (0x6a70 bytes) 2024-09-18T00:59:24.252 INFO:teuthology.orchestra.run.smithi042.stdout:4793 write 0xf20cf8 thru 0xf28156 (0x745f bytes) 2024-09-18T00:59:24.260 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:24.261 INFO:teuthology.orchestra.run.smithi042.stdout:4794 write 0x8054241 thru 0x8059a67 (0x5827 bytes) 2024-09-18T00:59:24.268 INFO:teuthology.orchestra.run.smithi042.stdout:4796 write 0xa986cf5 thru 0xa991b3b (0xae47 bytes) 2024-09-18T00:59:24.297 INFO:teuthology.orchestra.run.smithi042.stdout:4798 trunc from 0xee4b677 to 0x5c61d3d 2024-09-18T00:59:24.304 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:24.305+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:24.314 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:24.323 INFO:teuthology.orchestra.run.smithi042.stdout:4802 write 0x6336e0b thru 0x633eb96 (0x7d8c bytes) 2024-09-18T00:59:24.328 INFO:teuthology.orchestra.run.smithi042.stdout:4803 punch from 0x44241c3 to 0x4425939, (0x1776 bytes) 2024-09-18T00:59:24.328 INFO:teuthology.orchestra.run.smithi042.stdout:4804 read 0x379d7b4 thru 0x37ad253 (0xfaa0 bytes) 2024-09-18T00:59:24.360 INFO:teuthology.orchestra.run.smithi042.stdout:4805 writesame 0xcac715a thru 0xcad6e36 data_size 0x1(0xfcdd bytes) 2024-09-18T00:59:24.360 INFO:teuthology.orchestra.run.smithi042.stdout:4807 write 0x8967bde thru 0x896dac2 (0x5ee5 bytes) 2024-09-18T00:59:24.367 INFO:teuthology.orchestra.run.smithi042.stdout:4808 write 0x39870d1 thru 0x398e06f (0x6f9f bytes) 2024-09-18T00:59:24.375 INFO:teuthology.orchestra.run.smithi042.stdout:4809 write 0x7ce8ce9 thru 0x7cedc6c (0x4f84 bytes) 2024-09-18T00:59:24.382 INFO:teuthology.orchestra.run.smithi042.stdout:4810 read 0x66f2230 thru 0x66fc110 (0x9ee1 bytes) 2024-09-18T00:59:24.383 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:24.383 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:24.383 INFO:teuthology.orchestra.run.smithi042.stdout:4813 write 0xab7fb3c thru 0xab810d5 (0x159a bytes) 2024-09-18T00:59:24.392 INFO:teuthology.orchestra.run.smithi042.stdout:4814 writesame 0x764e612 thru 0x7652df7 data_size 0x1(0x47e6 bytes) 2024-09-18T00:59:24.401 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:24.401+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:24.405 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:24.405 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:24.420 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:24.420 INFO:teuthology.orchestra.run.smithi042.stdout:4816 write 0xa4b86d thru 0xa58156 (0xc8ea bytes) 2024-09-18T00:59:24.421 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:24.442 INFO:teuthology.orchestra.run.smithi042.stdout:4817 write 0xe23dea6 thru 0xe24b711 (0xd86c bytes) 2024-09-18T00:59:24.470 INFO:teuthology.orchestra.run.smithi042.stdout:4818 trunc from 0xe24b712 to 0x69aa4c6 2024-09-18T00:59:24.489 INFO:teuthology.orchestra.run.smithi042.stdout:4819 read 0x640fa07 thru 0x641afa6 (0xb5a0 bytes) 2024-09-18T00:59:24.494 INFO:teuthology.orchestra.run.smithi042.stdout:4820 trunc from 0x69aa4c6 to 0x4f917c0 2024-09-18T00:59:24.501 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:24.497+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:24.511 INFO:teuthology.orchestra.run.smithi042.stdout:4821 read 0x1b77ae8 thru 0x1b7ba4a (0x3f63 bytes) 2024-09-18T00:59:24.512 INFO:teuthology.orchestra.run.smithi042.stdout:4824 punch from 0x8df0ba to 0x8e8cd3, (0x9c19 bytes) 2024-09-18T00:59:24.512 INFO:teuthology.orchestra.run.smithi042.stdout:4825 write 0x13cc2b1 thru 0x13d32f5 (0x7045 bytes) 2024-09-18T00:59:24.516 INFO:teuthology.orchestra.run.smithi042.stdout:4826 punch from 0x4491a2d to 0x4497b33, (0x6106 bytes) 2024-09-18T00:59:24.540 INFO:teuthology.orchestra.run.smithi042.stdout:4827 write 0x9f8e54f thru 0x9f9de6f (0xf921 bytes) 2024-09-18T00:59:24.545 INFO:teuthology.orchestra.run.smithi042.stdout:4828 write 0x9c6acce thru 0x9c74664 (0x9997 bytes) 2024-09-18T00:59:24.550 INFO:teuthology.orchestra.run.smithi042.stdout:4829 read 0x5189e81 thru 0x518aac5 (0xc45 bytes) 2024-09-18T00:59:24.551 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:24.551 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:24.572 INFO:teuthology.orchestra.run.smithi042.stdout:4832 write 0xe8a5f17 thru 0xe8b2f12 (0xcffc bytes) 2024-09-18T00:59:24.576 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:24.577 INFO:teuthology.orchestra.run.smithi042.stdout:4834 read 0xd13153b thru 0xd13489b (0x3361 bytes) 2024-09-18T00:59:24.577 INFO:teuthology.orchestra.run.smithi042.stdout:4835 write 0x1992df8 thru 0x19987a9 (0x59b2 bytes) 2024-09-18T00:59:24.579 INFO:teuthology.orchestra.run.smithi042.stdout:4836 write 0x8ed3775 thru 0x8ee30c8 (0xf954 bytes) 2024-09-18T00:59:24.586 INFO:teuthology.orchestra.run.smithi042.stdout:4838 write 0x2ec5627 thru 0x2ed2a86 (0xd460 bytes) 2024-09-18T00:59:24.593 INFO:teuthology.orchestra.run.smithi042.stdout:4840 write 0x9373fcf thru 0x937ead4 (0xab06 bytes) 2024-09-18T00:59:24.596 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:24.596 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:24.599 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:24.603 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:24.601+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:24.612 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:24.612 INFO:teuthology.orchestra.run.smithi042.stdout:4843 trunc from 0xe8b2f13 to 0xa0d25ab 2024-09-18T00:59:24.632 INFO:teuthology.orchestra.run.smithi042.stdout:4844 read 0x4f4a9e8 thru 0x4f505cf (0x5be8 bytes) 2024-09-18T00:59:24.632 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:24.633 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:24.633 INFO:teuthology.orchestra.run.smithi042.stdout:4847 clone 38 order 24 su 2097152 sc 8 2024-09-18T00:59:24.703 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:24.701+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:24.734 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:24.734 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:24.748 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:24.832 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:24.833+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:24.906 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:24.901+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:24.910 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:24.910 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:24.923 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:25.006 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:25.005+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:25.087 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:25.087 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:25.101 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:25.106 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:25.105+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:25.207 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:25.205+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:25.228 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:25.228 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:25.243 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:25.328 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:25.329+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone37 intact 2024-09-18T00:59:25.408 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:25.405+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:25.411 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:25.411 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:25.425 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:25.516 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:25.517+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:25.598 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:25.598 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:25.611 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:25.609+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:25.613 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:25.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:25.718 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:25.717+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:25.720 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-18T00:59:25.747 INFO:teuthology.orchestra.run.smithi042.stdout:4848 write 0x2a3ed00 thru 0x2a4d264 (0xe565 bytes) 2024-09-18T00:59:25.779 INFO:teuthology.orchestra.run.smithi042.stdout:4849 trunc from 0xa0d25ab to 0x684388e 2024-09-18T00:59:25.807 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:25.808 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:25.812 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:25.809+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:25.836 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:25.842 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:25.842 INFO:teuthology.orchestra.run.smithi042.stdout:4851 read 0x6a8ee8 thru 0x6ab6e7 (0x2800 bytes) 2024-09-18T00:59:25.842 INFO:teuthology.orchestra.run.smithi042.stdout:4852 punch from 0x305c6d7 to 0x306a66e, (0xdf97 bytes) 2024-09-18T00:59:25.845 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:25.848 INFO:teuthology.orchestra.run.smithi042.stdout:4854 trunc from 0x684388e to 0xaa6f4c2 2024-09-18T00:59:25.855 INFO:teuthology.orchestra.run.smithi042.stdout:4855 write 0x8adbffe thru 0x8ae70a5 (0xb0a8 bytes) 2024-09-18T00:59:25.858 INFO:teuthology.orchestra.run.smithi042.stdout:4856 write 0x661e0ea thru 0x662a9c6 (0xc8dd bytes) 2024-09-18T00:59:25.860 INFO:teuthology.orchestra.run.smithi042.stdout:4857 read 0x46116b0 thru 0x46215ef (0xff40 bytes) 2024-09-18T00:59:25.860 INFO:teuthology.orchestra.run.smithi042.stdout:4859 read 0x1edf7b8 thru 0x1ee4278 (0x4ac1 bytes) 2024-09-18T00:59:25.861 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:25.861 INFO:teuthology.orchestra.run.smithi042.stdout:4862 read 0x524d1ce thru 0x52524bb (0x52ee bytes) 2024-09-18T00:59:25.861 INFO:teuthology.orchestra.run.smithi042.stdout:4863 punch from 0x104ddd2 to 0x1054282, (0x64b0 bytes) 2024-09-18T00:59:25.861 INFO:teuthology.orchestra.run.smithi042.stdout:4864 writesame 0x4acf291 thru 0x4adc1d5 data_size 0x17(0xcf45 bytes) 2024-09-18T00:59:25.864 INFO:teuthology.orchestra.run.smithi042.stdout:4865 read 0x9b649ab thru 0x9b69bae (0x5204 bytes) 2024-09-18T00:59:25.864 INFO:teuthology.orchestra.run.smithi042.stdout:4867 punch from 0x211fd39 to 0x2122703, (0x29ca bytes) 2024-09-18T00:59:25.865 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:25.865 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:25.876 INFO:teuthology.orchestra.run.smithi042.stdout:4868 writesame 0xdc9bce5 thru 0xdca4585 data_size 0x3(0x88a1 bytes) 2024-09-18T00:59:25.876 INFO:teuthology.orchestra.run.smithi042.stdout:4872 read 0x29eda85 thru 0x29eedc2 (0x133e bytes) 2024-09-18T00:59:25.877 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:25.877 INFO:teuthology.orchestra.run.smithi042.stdout:4875 write 0x5897c19 thru 0x589d000 (0x53e8 bytes) 2024-09-18T00:59:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:25.883 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:25.897 INFO:teuthology.orchestra.run.smithi042.stdout:4876 trunc from 0xdca4586 to 0xa67e4c 2024-09-18T00:59:25.901 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:25.901 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:25.912 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:25.909+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:25.918 INFO:teuthology.orchestra.run.smithi042.stdout:4877 read 0x366c24 thru 0x36ad25 (0x4102 bytes) 2024-09-18T00:59:25.918 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:25.918 INFO:teuthology.orchestra.run.smithi042.stdout:4880 read 0x301652 thru 0x3097ec (0x819b bytes) 2024-09-18T00:59:25.919 INFO:teuthology.orchestra.run.smithi042.stdout:4881 read 0x89d8e9 thru 0x8a334a (0x5a62 bytes) 2024-09-18T00:59:25.921 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:25.921 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:25.933 INFO:teuthology.orchestra.run.smithi042.stdout:4882 writesame 0x466b6a3 thru 0x4673a08 data_size 0x2(0x8366 bytes) 2024-09-18T00:59:25.940 INFO:teuthology.orchestra.run.smithi042.stdout:4883 trunc from 0x4673a09 to 0x7f3573d 2024-09-18T00:59:25.941 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:25.942 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:25.960 INFO:teuthology.orchestra.run.smithi042.stdout:4884 writesame 0xa84e57f thru 0xa851a46 data_size 0xc(0x34c8 bytes) 2024-09-18T00:59:25.961 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:25.962 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:25.969 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:25.969 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:25.978 INFO:teuthology.orchestra.run.smithi042.stdout:4885 writesame 0xea184ed thru 0xea23700 data_size 0x83(0xb214 bytes) 2024-09-18T00:59:25.978 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:25.979 INFO:teuthology.orchestra.run.smithi042.stdout:4887 write 0x4d7f685 thru 0x4d88148 (0x8ac4 bytes) 2024-09-18T00:59:25.981 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:25.981 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:25.985 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:26.008 INFO:teuthology.orchestra.run.smithi042.stdout:4888 trunc from 0xea23701 to 0x15a9b91 2024-09-18T00:59:26.028 INFO:teuthology.orchestra.run.smithi042.stdout:4890 write 0x1b6928c thru 0x1b77b7e (0xe8f3 bytes) 2024-09-18T00:59:26.029 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:26.029+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:26.030 INFO:teuthology.orchestra.run.smithi042.stdout:4891 read 0x1029468 thru 0x102c0f4 (0x2c8d bytes) 2024-09-18T00:59:26.044 INFO:teuthology.orchestra.run.smithi042.stdout:4892 trunc from 0x1b77b7f to 0xa78361f 2024-09-18T00:59:26.053 INFO:teuthology.orchestra.run.smithi042.stdout:4893 write 0x159c68d thru 0x159ec35 (0x25a9 bytes) 2024-09-18T00:59:26.058 INFO:teuthology.orchestra.run.smithi042.stdout:4894 trunc from 0xa78361f to 0xc842b90 2024-09-18T00:59:26.068 INFO:teuthology.orchestra.run.smithi042.stdout:4895 read 0x649ec2f thru 0x64a21e7 (0x35b9 bytes) 2024-09-18T00:59:26.068 INFO:teuthology.orchestra.run.smithi042.stdout:4896 write 0x66f94b8 thru 0x6706644 (0xd18d bytes) 2024-09-18T00:59:26.070 INFO:teuthology.orchestra.run.smithi042.stdout:4897 read 0xb816c09 thru 0xb81d84e (0x6c46 bytes) 2024-09-18T00:59:26.071 INFO:teuthology.orchestra.run.smithi042.stdout:4899 read 0x20bac8d thru 0x20c6849 (0xbbbd bytes) 2024-09-18T00:59:26.071 INFO:teuthology.orchestra.run.smithi042.stdout:4900 punch from 0x568825b to 0x568e75c, (0x6501 bytes) 2024-09-18T00:59:26.071 INFO:teuthology.orchestra.run.smithi042.stdout:4901 read 0x1354965 thru 0x1359480 (0x4b1c bytes) 2024-09-18T00:59:26.072 INFO:teuthology.orchestra.run.smithi042.stdout:4902 write 0xc5ee66d thru 0xc5f4a56 (0x63ea bytes) 2024-09-18T00:59:26.077 INFO:teuthology.orchestra.run.smithi042.stdout:4903 writesame 0xb2e649 thru 0xb35d1d data_size 0x1(0x76d5 bytes) 2024-09-18T00:59:26.096 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.096 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.096 INFO:teuthology.orchestra.run.smithi042.stdout:4906 write 0x505c441 thru 0x506a609 (0xe1c9 bytes) 2024-09-18T00:59:26.100 INFO:teuthology.orchestra.run.smithi042.stdout:4907 trunc from 0xc842b90 to 0xb8fb986 2024-09-18T00:59:26.112 INFO:teuthology.orchestra.run.smithi042.stdout:4908 read 0x4283762 thru 0x429227d (0xeb1c bytes) 2024-09-18T00:59:26.113 INFO:teuthology.orchestra.run.smithi042.stdout:4909 write 0x7096523 thru 0x709f719 (0x91f7 bytes) 2024-09-18T00:59:26.113 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:26.113 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:26.113 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:26.113 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:26.113 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:26.114 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:26.113+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:26.115 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.116 INFO:teuthology.orchestra.run.smithi042.stdout:4911 punch from 0x64e3ca7 to 0x64eb95d, (0x7cb6 bytes) 2024-09-18T00:59:26.116 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.128 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:26.133 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6, 7] out_osds: [2, 3, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:59:26.133 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:59:26.134 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [1, 4, 5, 6, 7] 2024-09-18T00:59:26.134 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-18T00:59:26.135 INFO:teuthology.orchestra.run.smithi042.stdout:4914 trunc from 0xb8fb986 to 0x3cc31a8 2024-09-18T00:59:26.162 INFO:teuthology.orchestra.run.smithi042.stdout:4915 write 0x5358b73 thru 0x53620b8 (0x9546 bytes) 2024-09-18T00:59:26.164 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:59:26.183 INFO:teuthology.orchestra.run.smithi042.stdout:4916 write 0xb723964 thru 0xb726c47 (0x32e4 bytes) 2024-09-18T00:59:26.188 INFO:teuthology.orchestra.run.smithi042.stdout:4917 write 0xee4af9 thru 0xef2ead (0xe3b5 bytes) 2024-09-18T00:59:26.192 INFO:teuthology.orchestra.run.smithi042.stdout:4918 read 0x4abef9e thru 0x4ac32b2 (0x4315 bytes) 2024-09-18T00:59:26.204 INFO:teuthology.orchestra.run.smithi042.stdout:4919 write 0xe1a8f64 thru 0xe1b4d5e (0xbdfb bytes) 2024-09-18T00:59:26.215 INFO:teuthology.orchestra.run.smithi042.stdout:4920 write 0xe3ac346 thru 0xe3b49c5 (0x8680 bytes) 2024-09-18T00:59:26.215 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:26.213+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:26.219 INFO:teuthology.orchestra.run.smithi042.stdout:4921 write 0xc57cac0 thru 0xc57f69a (0x2bdb bytes) 2024-09-18T00:59:26.222 INFO:teuthology.orchestra.run.smithi042.stdout:4922 write 0xbb62ecd thru 0xbb685ea (0x571e bytes) 2024-09-18T00:59:26.227 INFO:teuthology.orchestra.run.smithi042.stdout:4923 punch from 0xa1bf56e to 0xa1c3a65, (0x44f7 bytes) 2024-09-18T00:59:26.227 INFO:teuthology.orchestra.run.smithi042.stdout:4924 writesame 0x53b0d5b thru 0x53b892d data_size 0x1(0x7bd3 bytes) 2024-09-18T00:59:26.248 INFO:teuthology.orchestra.run.smithi042.stdout:4925 punch from 0x6a8d6c0 to 0x6a9ab63, (0xd4a3 bytes) 2024-09-18T00:59:26.266 INFO:teuthology.orchestra.run.smithi042.stdout:4926 trunc from 0xe3b49c6 to 0x2b6ff85 2024-09-18T00:59:26.289 INFO:teuthology.orchestra.run.smithi042.stdout:4927 read 0x29a0c1 thru 0x2a350b (0x944b bytes) 2024-09-18T00:59:26.297 INFO:teuthology.orchestra.run.smithi042.stdout:4928 trunc from 0x2b6ff85 to 0x65e7dd8 2024-09-18T00:59:26.305 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.306 INFO:teuthology.orchestra.run.smithi042.stdout:4930 writesame 0x5f39616 thru 0x5f3e4d8 data_size 0x2f(0x4ec3 bytes) 2024-09-18T00:59:26.307 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:26.307 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:26.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:26.307 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 101.30302997699999, 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:59:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:59:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 101.30297172, 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:59:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:59:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 101.302913281, 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:59:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 101.30285206400001, 2024-09-18T00:59:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:59:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 101.302761649, 2024-09-18T00:59:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:59:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:59:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:59:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:59:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:59:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:59:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:59:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:59:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:59:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:59:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:59:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:59:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:59:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:59:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:59:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:59:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 101.302690716, 2024-09-18T00:59:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:59:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:59:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 101.30263110999999, 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:59:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:59:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:59:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:59:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:59:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:59:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 101.30257174, 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:59:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 101.302509951, 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:59:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:59:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:59:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:59:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:59:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:59:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:59:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:59:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:59:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:59:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 101.302377128, 2024-09-18T00:59:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:59:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:59:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:59:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:59:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:59:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:59:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:59:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:59:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:59:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:59:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:59:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:59:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:59:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:59:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:59:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:59:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 97.747159651000004, 2024-09-18T00:59:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T00:59:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T00:59:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T00:59:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T00:59:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T00:59:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T00:59:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T00:59:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T00:59:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T00:59:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T00:59:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T00:59:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T00:59:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T00:59:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T00:59:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T00:59:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T00:59:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T00:59:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T00:59:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T00:59:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T00:59:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T00:59:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T00:59:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T00:59:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T00:59:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 97.746659527000006, 2024-09-18T00:59:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T00:59:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T00:59:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T00:59:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T00:59:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T00:59:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T00:59:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 97.746629018999997, 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T00:59:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T00:59:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 97.746604353999999, 2024-09-18T00:59:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T00:59:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T00:59:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 97.746575821999997, 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T00:59:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 97.746242531000007, 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 97.746200680000001, 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:59:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 97.745163184000006, 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.092875179000004, 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T00:59:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T00:59:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T00:59:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:138250 3.c 3:32730bfd:::rbd_data.117c1c3044f5.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:21.758914+0000", 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5471773710000001, 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060687680000000001, 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 138250 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758917+0000", 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5550000000000001e-06 2024-09-18T00:59:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758925+0000", 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5900000000000008e-06 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758947+0000", 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1197000000000001e-05 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758957+0000", 2024-09-18T00:59:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0137e-05 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759094+0000", 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013775300000000001 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759246+0000", 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015127900000000001 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.770839+0000", 2024-09-18T00:59:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011593538 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819413+0000", 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048573660999999997 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819467+0000", 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3795999999999998e-05 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819529+0000", 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2675999999999994e-05 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819602+0000", 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2497999999999996e-05 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:26.420 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:26.422 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:26.423 INFO:teuthology.orchestra.run.smithi042.stdout:4933 read 0x108eed5 thru 0x109d32d (0xe459 bytes) 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:4934 trunc from 0x65e7dd8 to 0x7c6aa18 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:4935 read 0x13f3c78 thru 0x13fc674 (0x89fd bytes) 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:4936 writesame 0xa7ec738 thru 0xa7fb8b7 data_size 0x1c(0xf180 bytes) 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:4938 punch from 0x78b7a54 to 0x78c51f9, (0xd7a5 bytes) 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:4939 trunc from 0xa7fb8b8 to 0x5a82156 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:4941 trunc from 0x5a82156 to 0x250700f 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:4947 read 0xff8735 thru 0xffb814 (0x30e0 bytes) 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:4949 read 0x29f9e4 thru 0x2ae476 (0xea93 bytes) 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:4950 write 0x8af460 thru 0x8bacb9 (0xb85a bytes) 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:4952 trunc from 0x250700f to 0x82b1dfd 2024-09-18T00:59:26.424 INFO:teuthology.orchestra.run.smithi042.stdout:4953 writesame 0x4793317 thru 0x479bfef data_size 0x33(0x8cd9 bytes) 2024-09-18T00:59:26.425 INFO:teuthology.orchestra.run.smithi042.stdout:4954 write 0xa719cae thru 0xa71edfa (0x514d bytes) 2024-09-18T00:59:26.425 INFO:teuthology.orchestra.run.smithi042.stdout:4955 read 0x4256fc0 thru 0x4261f58 (0xaf99 bytes) 2024-09-18T00:59:26.425 INFO:teuthology.orchestra.run.smithi042.stdout:4958 writesame 0x4838261 thru 0x4840ea0 data_size 0x66(0x8c40 bytes) 2024-09-18T00:59:26.425 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:26.421+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:26.425 INFO:teuthology.orchestra.run.smithi042.stdout:4959 trunc from 0xa71edfb to 0x5dbe05 2024-09-18T00:59:26.475 INFO:teuthology.orchestra.run.smithi042.stdout:4960 read 0x42a536 thru 0x435b5e (0xb629 bytes) 2024-09-18T00:59:26.480 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-18T00:59:26.477+0000 7fdb35eeb640 -1 mon.a@0(leader).osd e113 definitely_dead 0 2024-09-18T00:59:26.491 INFO:teuthology.orchestra.run.smithi042.stdout:4961 trunc from 0x5dbe05 to 0x9e4c961 2024-09-18T00:59:26.504 INFO:teuthology.orchestra.run.smithi042.stdout:4962 trunc from 0x9e4c961 to 0xa74d76a 2024-09-18T00:59:26.522 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:26.521+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:26.539 INFO:teuthology.orchestra.run.smithi042.stdout:4965 writesame 0xeb2f814 thru 0xeb31713 data_size 0x20(0x1f00 bytes) 2024-09-18T00:59:26.549 INFO:teuthology.orchestra.run.smithi042.stdout:4968 trunc from 0xeb31714 to 0x82760bb 2024-09-18T00:59:26.561 INFO:teuthology.orchestra.run.smithi042.stdout:4971 writesame 0x35a673d thru 0x35ae4e0 data_size 0x2b(0x7da4 bytes) 2024-09-18T00:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.565 INFO:teuthology.orchestra.run.smithi042.stdout:4973 read 0xb1cfd thru 0xb6276 (0x457a bytes) 2024-09-18T00:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout:4974 read 0x5aa20ec thru 0x5ab057e (0xe493 bytes) 2024-09-18T00:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout:4976 read 0x46c28b5 thru 0x46c728f (0x49db bytes) 2024-09-18T00:59:26.567 INFO:teuthology.orchestra.run.smithi042.stdout:4977 read 0x1db95b thru 0x1e022e (0x48d4 bytes) 2024-09-18T00:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.568 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.575 INFO:teuthology.orchestra.run.smithi042.stdout:4988 trunc from 0x82760bb to 0x3c203c8 2024-09-18T00:59:26.586 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:26.592 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.609 INFO:teuthology.orchestra.run.smithi042.stdout:4991 writesame 0x737a4ba thru 0x73834d2 data_size 0x1(0x9019 bytes) 2024-09-18T00:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout:4992 read 0x4cad7b3 thru 0x4caddf0 (0x63e bytes) 2024-09-18T00:59:26.612 INFO:teuthology.orchestra.run.smithi042.stdout:4994 write 0x4bb50c7 thru 0x4bb908b (0x3fc5 bytes) 2024-09-18T00:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout:4995 punch from 0xc6aeae to 0xc6f724, (0x4876 bytes) 2024-09-18T00:59:26.619 INFO:teuthology.orchestra.run.smithi042.stdout:4996 writesame 0x322bb92 thru 0x32342fc data_size 0x1(0x876b bytes) 2024-09-18T00:59:26.623 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:26.621+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:26.666 INFO:teuthology.orchestra.run.smithi042.stdout:4998 read 0x70ee1ad thru 0x70faaf3 (0xc947 bytes) 2024-09-18T00:59:26.666 INFO:teuthology.orchestra.run.smithi042.stdout:4999 punch from 0x368be9c to 0x368ca0b, (0xb6f bytes) 2024-09-18T00:59:26.666 INFO:teuthology.orchestra.run.smithi042.stdout:5000 writesame 0x5146adb thru 0x5147116 data_size 0xe(0x63c bytes) 2024-09-18T00:59:26.669 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.670 INFO:teuthology.orchestra.run.smithi042.stdout:5002 punch from 0x4f9b2b1 to 0x4fa0b02, (0x5851 bytes) 2024-09-18T00:59:26.670 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.693 INFO:teuthology.orchestra.run.smithi042.stdout:5004 write 0xc9f12d8 thru 0xc9f180a (0x533 bytes) 2024-09-18T00:59:26.699 INFO:teuthology.orchestra.run.smithi042.stdout:5005 punch from 0x5da9035 to 0x5db13c8, (0x8393 bytes) 2024-09-18T00:59:26.699 INFO:teuthology.orchestra.run.smithi042.stdout:5006 write 0x7791d6d thru 0x779b601 (0x9895 bytes) 2024-09-18T00:59:26.706 INFO:teuthology.orchestra.run.smithi042.stdout:5008 writesame 0x75f32ba thru 0x75f8204 data_size 0x1(0x4f4b bytes) 2024-09-18T00:59:26.712 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:26.713 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:26.713 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:26.713 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:26.713 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:26.724 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:26.721+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:26.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:26.752 INFO:teuthology.orchestra.run.smithi042.stdout:5009 write 0xe5446bd thru 0xe54cde2 (0x8726 bytes) 2024-09-18T00:59:26.760 INFO:teuthology.orchestra.run.smithi042.stdout:5011 write 0x50fea8f thru 0x5102ba1 (0x4113 bytes) 2024-09-18T00:59:26.776 INFO:teuthology.orchestra.run.smithi042.stdout:5013 trunc from 0xe54cde3 to 0x650da22 2024-09-18T00:59:26.801 INFO:teuthology.orchestra.run.smithi042.stdout:5015 read 0xd8e843 thru 0xd9e73b (0xfef9 bytes) 2024-09-18T00:59:26.811 INFO:teuthology.orchestra.run.smithi042.stdout:5016 trunc from 0x650da22 to 0x3c985d 2024-09-18T00:59:26.824 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:26.825+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:26.839 INFO:teuthology.orchestra.run.smithi042.stdout:5017 read 0x52751 thru 0x5b7b3 (0x9063 bytes) 2024-09-18T00:59:26.840 INFO:teuthology.orchestra.run.smithi042.stdout:5020 read 0x3b9d0f thru 0x3c5a9c (0xbd8e bytes) 2024-09-18T00:59:26.851 INFO:teuthology.orchestra.run.smithi042.stdout:5022 trunc from 0x3c985d to 0x546e7ac 2024-09-18T00:59:26.862 INFO:teuthology.orchestra.run.smithi042.stdout:5023 punch from 0x3064ea7 to 0x306ccab, (0x7e04 bytes) 2024-09-18T00:59:26.869 INFO:teuthology.orchestra.run.smithi042.stdout:5024 trunc from 0x546e7ac to 0x2303b31 2024-09-18T00:59:26.885 INFO:teuthology.orchestra.run.smithi042.stdout:5027 write 0x3fafde thru 0x406650 (0xb673 bytes) 2024-09-18T00:59:26.891 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:26.891 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:26.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:26.891 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:26.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.891 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:26.891 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:26.891 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.823402528000003, 2024-09-18T00:59:26.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:59:26.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:59:26.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T00:59:26.897 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.930868708999999, 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T00:59:26.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T00:59:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.181801919000002, 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T00:59:26.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T00:59:26.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.171427463000001, 2024-09-18T00:59:26.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:26.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T00:59:26.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.911 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.171354781000002, 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T00:59:26.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.171323914999999, 2024-09-18T00:59:26.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T00:59:26.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.920 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.171271140999998, 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.921 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T00:59:26.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.171244975, 2024-09-18T00:59:26.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.171219820000001, 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T00:59:26.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.935 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.935 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:26.935 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.170796508999999, 2024-09-18T00:59:26.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T00:59:26.935 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:26.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.939 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.170758130999999, 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T00:59:26.943 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.170733366, 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.944 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.163938602000002, 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T00:59:26.948 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T00:59:26.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.949 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:26.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:26.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.955 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.163913002000001, 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T00:59:26.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.163889658999999, 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.960 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.964 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.16386722, 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:26.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T00:59:26.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.163846447000001, 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T00:59:26.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.973 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.163825420999999, 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T00:59:26.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.163784839000002, 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T00:59:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.163750740000001, 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:26.987 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:26.988 INFO:teuthology.orchestra.run.smithi042.stdout:5028 punch from 0x109607d to 0x10a4493, (0xe416 bytes) 2024-09-18T00:59:26.988 INFO:teuthology.orchestra.run.smithi042.stdout:5029 write 0xc4e7f1 thru 0xc572d4 (0x8ae4 bytes) 2024-09-18T00:59:26.988 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.988 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.988 INFO:teuthology.orchestra.run.smithi042.stdout:5033 read 0x38a4cc thru 0x392cf7 (0x882c bytes) 2024-09-18T00:59:26.989 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:26.990 INFO:teuthology.orchestra.run.smithi042.stdout:5034 writesame 0x4d7f306 thru 0x4d839e7 data_size 0x2b(0x46e2 bytes) 2024-09-18T00:59:26.990 INFO:teuthology.orchestra.run.smithi042.stdout:5035 writesame 0xc669b56 thru 0xc66d335 data_size 0x30(0x37e0 bytes) 2024-09-18T00:59:26.990 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.990 INFO:teuthology.orchestra.run.smithi042.stdout:5037 read 0x2007297 thru 0x2013b28 (0xc892 bytes) 2024-09-18T00:59:26.990 INFO:teuthology.orchestra.run.smithi042.stdout:5038 read 0x2b75047 thru 0x2b78bf7 (0x3bb1 bytes) 2024-09-18T00:59:26.991 INFO:teuthology.orchestra.run.smithi042.stdout:5039 write 0x46d46f1 thru 0x46d8070 (0x3980 bytes) 2024-09-18T00:59:26.991 INFO:teuthology.orchestra.run.smithi042.stdout:5041 write 0xb0d0926 thru 0xb0d0ae9 (0x1c4 bytes) 2024-09-18T00:59:26.991 INFO:teuthology.orchestra.run.smithi042.stdout:5042 read 0x7751ed9 thru 0x7754873 (0x299b bytes) 2024-09-18T00:59:26.991 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:26.991 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:26.985+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:26.995 INFO:teuthology.orchestra.run.smithi042.stdout:5047 writesame 0xd8308f0 thru 0xd837eab data_size 0x37(0x75bc bytes) 2024-09-18T00:59:26.996 INFO:teuthology.orchestra.run.smithi042.stdout:5048 read 0x21ea87f thru 0x21f57f3 (0xaf75 bytes) 2024-09-18T00:59:27.028 INFO:teuthology.orchestra.run.smithi042.stdout:5049 trunc from 0xd837eac to 0xda8274 2024-09-18T00:59:27.059 INFO:teuthology.orchestra.run.smithi042.stdout:5051 read 0x53bca9 thru 0x544f14 (0x926c bytes) 2024-09-18T00:59:27.076 INFO:teuthology.orchestra.run.smithi042.stdout:5053 write 0x18e35b6 thru 0x18e820f (0x4c5a bytes) 2024-09-18T00:59:27.088 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:27.085+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:27.122 INFO:teuthology.orchestra.run.smithi042.stdout:5054 write 0xc032e7a thru 0xc0357dc (0x2963 bytes) 2024-09-18T00:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:27.127 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:27.129 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:27.146 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:27.153 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:59:27.154 INFO:teuthology.orchestra.run.smithi042.stdout:5056 trunc from 0xc0357dd to 0xd1b216 2024-09-18T00:59:27.157 INFO:teuthology.orchestra.run.smithi042.stderr:marked out osd.7. 2024-09-18T00:59:27.226 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:27.225+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:27.274 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:27.274 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:27.275 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:27.275 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:27.275 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:27.289 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:27.290 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:27.289+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:27.390 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:27.389+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:27.459 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:27.459 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:27.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:27.459 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:27.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.459 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 124.16783623000001, 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:59:27.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:59:27.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:27.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.971495176999994, 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:59:27.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.971473739999993, 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:27.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.971378412999996, 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:59:27.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:27.479 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.971352973999998, 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:59:27.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:59:27.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.971330451, 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:27.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:59:27.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:59:27.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.971307010999993, 2024-09-18T00:59:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:27.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:59:27.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.971283201000006, 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.493 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:59:27.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.971260835999999, 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:59:27.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:59:27.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:59:27.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:59:27.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:59:27.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.502 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.502 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.502 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:27.502 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.971234218999996, 2024-09-18T00:59:27.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:59:27.502 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:59:27.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.503 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:27.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:27.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:59:27.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:59:27.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:59:27.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:59:27.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:59:27.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:59:27.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:59:27.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:59:27.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:59:27.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:59:27.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.508 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.508 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:27.508 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.967082447999999, 2024-09-18T00:59:27.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:59:27.508 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:59:27.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.509 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:27.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:27.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:59:27.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:59:27.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:59:27.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:59:27.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:59:27.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:59:27.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.513 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.513 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.513 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:27.513 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.967061705000006, 2024-09-18T00:59:27.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:59:27.513 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:27.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:59:27.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:59:27.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:59:27.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:59:27.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:59:27.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:59:27.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:59:27.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:59:27.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:59:27.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:59:27.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.518 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.518 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.518 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.967016139999998, 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:27.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:27.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:59:27.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:59:27.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:59:27.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:59:27.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:59:27.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:59:27.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.524 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.524 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:27.524 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.966989799000004, 2024-09-18T00:59:27.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:59:27.524 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:59:27.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:59:27.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:59:27.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:59:27.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:59:27.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.966967128999997, 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:27.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:27.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:59:27.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:59:27.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:59:27.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:59:27.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:59:27.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:59:27.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:59:27.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:59:27.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:59:27.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.534 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.534 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.966948755999994, 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.535 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:27.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:27.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:59:27.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:59:27.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:59:27.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:59:27.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:59:27.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:59:27.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:59:27.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:59:27.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:59:27.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.540 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:27.540 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.966928968000005, 2024-09-18T00:59:27.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:59:27.541 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:59:27.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.541 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:27.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:27.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:59:27.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:59:27.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:59:27.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:59:27.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:59:27.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:59:27.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:59:27.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:59:27.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:59:27.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:59:27.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.546 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:27.546 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:27.546 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.966902098000006, 2024-09-18T00:59:27.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:59:27.546 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:27.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:27.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:59:27.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:59:27.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:27.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:59:27.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:59:27.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:59:27.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:59:27.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:59:27.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:59:27.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.551 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T00:59:27.551 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:27.552 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.890711629000002, 2024-09-18T00:59:27.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T00:59:27.552 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:27.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.552 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T00:59:27.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.552 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T00:59:27.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T00:59:27.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T00:59:27.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T00:59:27.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T00:59:27.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:27.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T00:59:27.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T00:59:27.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:27.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T00:59:27.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T00:59:27.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:27.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.558 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.479646533, 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.559 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T00:59:27.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:27.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T00:59:27.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T00:59:27.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:27.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T00:59:27.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T00:59:27.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:27.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T00:59:27.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T00:59:27.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:27.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T00:59:27.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T00:59:27.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:27.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:27.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:27.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:27.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T00:59:27.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T00:59:27.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:27.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:27.566 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:27.568 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:27.568 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:27.565+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:27.669 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:27.665+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:27.741 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:27.742 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:27.742 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:27.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:27.798 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:27.797+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:27.870 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:27.869+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:27.925 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:27.926 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:27.941 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:27.971 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:27.969+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:28.006 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:28.074 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:28.073+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:28.119 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:28.119 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:28.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:28.119 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:28.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.119 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.119 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:28.119 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.039050516, 2024-09-18T00:59:28.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:59:28.119 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:59:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.038120925, 2024-09-18T00:59:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:59:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:59:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.038106865, 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:59:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.03681629, 2024-09-18T00:59:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:59:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:59:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.036795818, 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:59:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:59:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:59:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:59:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.03400136499999, 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.033986181, 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:59:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:59:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.033963235, 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:59:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:59:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.029785596, 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:59:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:59:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:59:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.029764379, 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:59:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:59:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:59:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:59:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.029742996, 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:59:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.025843142, 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:59:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.025828941, 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:59:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:59:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:59:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:59:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:59:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.025810904, 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:59:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.02579683800001, 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:59:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:59:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:59:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.021771487, 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:59:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.021757475, 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:59:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:59:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:59:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.021743511, 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:59:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:59:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 99.259885557000004, 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:59:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 81.134409185999999, 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:59:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:59:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:59:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:59:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:28.211 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:28.213 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:28.213 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:28.213 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:28.214 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:28.214 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:28.214 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:28.215 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:28.215 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:28.209+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:28.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:28.228 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:28.249 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:28.249 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:28.270 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:28.270 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:28.292 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:28.313 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:28.313+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:28.314 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:28.367 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:28.454 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:28.453+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:28.516 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:28.513+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:28.537 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:28.537 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:28.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:28.617 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:28.617+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:28.718 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:28.717+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:28.722 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:28.722 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:28.737 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:28.818 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:28.817+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:28.867 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:28.868 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:28.882 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:28.924 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:28.925+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:29.020 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:29.021+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:29.050 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:29.050 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:29.064 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:29.152 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:29.153+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:29.179 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:59:29.222 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:29.221+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:29.236 INFO:teuthology.orchestra.run.smithi042.stdout:5057 write 0x884049 thru 0x89339a (0xf352 bytes) 2024-09-18T00:59:29.253 INFO:teuthology.orchestra.run.smithi042.stdout:5058 trunc from 0xd1b216 to 0x7f0d26f 2024-09-18T00:59:29.262 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:29.262 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:29.263 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:29.264 INFO:teuthology.orchestra.run.smithi042.stdout:5060 writesame 0x2469724 thru 0x24774b6 data_size 0x1(0xdd93 bytes) 2024-09-18T00:59:29.288 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:29.303 INFO:teuthology.orchestra.run.smithi042.stdout:5061 write 0x6454bd9 thru 0x6460756 (0xbb7e bytes) 2024-09-18T00:59:29.307 INFO:teuthology.orchestra.run.smithi042.stdout:5062 punch from 0x51bd613 to 0x51c191b, (0x4308 bytes) 2024-09-18T00:59:29.322 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:29.321+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:29.324 INFO:teuthology.orchestra.run.smithi042.stdout:5064 write 0xb6ad9fa thru 0xb6bd124 (0xf72b bytes) 2024-09-18T00:59:29.331 INFO:teuthology.orchestra.run.smithi042.stdout:5065 write 0x1b39732 thru 0x1b3df2c (0x47fb bytes) 2024-09-18T00:59:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:5067 read 0x5d7e19e thru 0x5d8e10c (0xff6f bytes) 2024-09-18T00:59:29.337 INFO:teuthology.orchestra.run.smithi042.stdout:5068 punch from 0x67ea1fb to 0x67ec3c4, (0x21c9 bytes) 2024-09-18T00:59:29.339 INFO:teuthology.orchestra.run.smithi042.stdout:5069 trunc from 0xb6bd125 to 0xa292d68 2024-09-18T00:59:29.348 INFO:teuthology.orchestra.run.smithi042.stdout:5071 read 0x227e43d thru 0x2288e35 (0xa9f9 bytes) 2024-09-18T00:59:29.349 INFO:teuthology.orchestra.run.smithi042.stdout:5072 read 0x3a25708 thru 0x3a34893 (0xf18c bytes) 2024-09-18T00:59:29.349 INFO:teuthology.orchestra.run.smithi042.stdout:5073 punch from 0x6e9c360 to 0x6ea3bfa, (0x789a bytes) 2024-09-18T00:59:29.349 INFO:teuthology.orchestra.run.smithi042.stdout:5074 read 0xa17b103 thru 0xa183fe3 (0x8ee1 bytes) 2024-09-18T00:59:29.349 INFO:teuthology.orchestra.run.smithi042.stdout:5076 read 0x74bfc09 thru 0x74ce83a (0xec32 bytes) 2024-09-18T00:59:29.350 INFO:teuthology.orchestra.run.smithi042.stdout:5077 read 0x544ba thru 0x59262 (0x4da9 bytes) 2024-09-18T00:59:29.350 INFO:teuthology.orchestra.run.smithi042.stdout:5079 write 0x274c03d thru 0x2751ab4 (0x5a78 bytes) 2024-09-18T00:59:29.352 INFO:teuthology.orchestra.run.smithi042.stdout:5080 punch from 0x5a2df05 to 0x5a3890f, (0xaa0a bytes) 2024-09-18T00:59:29.352 INFO:teuthology.orchestra.run.smithi042.stdout:5081 write 0x3d21b2a thru 0x3d2adb0 (0x9287 bytes) 2024-09-18T00:59:29.354 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:29.355 INFO:teuthology.orchestra.run.smithi042.stdout:5083 write 0xeeb312 thru 0xeebeda (0xbc9 bytes) 2024-09-18T00:59:29.359 INFO:teuthology.orchestra.run.smithi042.stdout:5084 writesame 0x3dbd7fe thru 0x3dc5acb data_size 0x3(0x82ce bytes) 2024-09-18T00:59:29.381 INFO:teuthology.orchestra.run.smithi042.stdout:5085 write 0xddff064 thru 0xde0390a (0x48a7 bytes) 2024-09-18T00:59:29.387 INFO:teuthology.orchestra.run.smithi042.stdout:5086 write 0x49f3a53 thru 0x4a027b0 (0xed5e bytes) 2024-09-18T00:59:29.389 INFO:teuthology.orchestra.run.smithi042.stdout:5087 writesame 0x6c42223 thru 0x6c507dc data_size 0x5(0xe5ba bytes) 2024-09-18T00:59:29.398 INFO:teuthology.orchestra.run.smithi042.stdout:5088 read 0x1b00960 thru 0x1b0eeee (0xe58f bytes) 2024-09-18T00:59:29.401 INFO:teuthology.orchestra.run.smithi042.stdout:5090 trunc from 0xde0390b to 0xc35c90c 2024-09-18T00:59:29.417 INFO:teuthology.orchestra.run.smithi042.stdout:5091 trunc from 0xc35c90c to 0x96a779a 2024-09-18T00:59:29.423 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:29.421+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:29.426 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:29.426 INFO:teuthology.orchestra.run.smithi042.stdout:5095 punch from 0x74d3530 to 0x74df0c9, (0xbb99 bytes) 2024-09-18T00:59:29.426 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:29.434 INFO:teuthology.orchestra.run.smithi042.stdout:5098 write 0x9c33db7 thru 0x9c36c30 (0x2e7a bytes) 2024-09-18T00:59:29.436 INFO:teuthology.orchestra.run.smithi042.stdout:5099 writesame 0x5123d20 thru 0x512bcdd data_size 0x2(0x7fbe bytes) 2024-09-18T00:59:29.448 INFO:teuthology.orchestra.run.smithi042.stdout:5100 punch from 0x33d6030 to 0x33e28d3, (0xc8a3 bytes) 2024-09-18T00:59:29.448 INFO:teuthology.orchestra.run.smithi042.stdout:5102 punch from 0x8907e9f to 0x890facc, (0x7c2d bytes) 2024-09-18T00:59:29.448 INFO:teuthology.orchestra.run.smithi042.stdout:5103 writesame 0x188e56f thru 0x1897924 data_size 0x4a(0x93b6 bytes) 2024-09-18T00:59:29.450 INFO:teuthology.orchestra.run.smithi042.stdout:5104 read 0x9a0f1f3 thru 0x9a1ade3 (0xbbf1 bytes) 2024-09-18T00:59:29.450 INFO:teuthology.orchestra.run.smithi042.stdout:5105 write 0x983636 thru 0x98f398 (0xbd63 bytes) 2024-09-18T00:59:29.452 INFO:teuthology.orchestra.run.smithi042.stdout:5107 writesame 0x6578eac thru 0x6586a4e data_size 0x1(0xdba3 bytes) 2024-09-18T00:59:29.492 INFO:teuthology.orchestra.run.smithi042.stdout:5108 read 0x184f3dc thru 0x1857d9e (0x89c3 bytes) 2024-09-18T00:59:29.493 INFO:teuthology.orchestra.run.smithi042.stdout:5110 write 0x705dd85 thru 0x70694e4 (0xb760 bytes) 2024-09-18T00:59:29.495 INFO:teuthology.orchestra.run.smithi042.stdout:5111 write 0x49b6ab9 thru 0x49c6943 (0xfe8b bytes) 2024-09-18T00:59:29.497 INFO:teuthology.orchestra.run.smithi042.stdout:5112 write 0x9c0d341 thru 0x9c1cf26 (0xfbe6 bytes) 2024-09-18T00:59:29.503 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:29.503 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:29.511 INFO:teuthology.orchestra.run.smithi042.stdout:5114 trunc from 0x9c36c31 to 0xe681037 2024-09-18T00:59:29.520 INFO:teuthology.orchestra.run.smithi042.stdout:5116 write 0xe0cbc4f thru 0xe0dbc1c (0xffce bytes) 2024-09-18T00:59:29.524 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:29.521+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:29.525 INFO:teuthology.orchestra.run.smithi042.stdout:5117 clone 39 order 25 su 131072 sc 15 2024-09-18T00:59:29.526 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:29.624 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:29.625+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:29.712 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:29.712 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:29.725 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:29.725+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:29.730 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:29.826 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:29.825+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:29.914 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:29.914 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:29.928 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:29.925+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:29.930 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:30.027 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:30.025+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:30.113 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:30.113 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:30.128 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:30.125+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:30.130 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:30.166 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:59:30.229 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:30.229+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:30.300 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:30.301 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:30.316 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:30.333 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:30.364 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:30.365+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:30.430 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:30.429+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:30.486 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:30.486 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:30.493 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:30.494 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:30.500 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:30.510 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:30.510 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:30.526 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:30.527 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:30.531 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:30.529+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:30.543 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:30.562 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:30.582 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:30.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:30.603 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:30.603 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:30.618 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:30.618 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:30.623 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:30.624 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:30.632 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:30.629+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:30.636 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:30.733 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:30.729+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:30.802 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:30.802 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:30.802 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:30.802 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:30.802 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:30.816 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:30.833 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:30.829+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:30.934 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:30.933+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:30.938 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:30.938 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:30.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:30.938 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:30.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.938 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:30.938 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:30.938 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 105.931967493, 2024-09-18T00:59:30.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T00:59:30.938 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.938 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T00:59:30.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.942 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 105.931909236, 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.943 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T00:59:30.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 105.931850797, 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T00:59:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T00:59:30.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 105.93178958, 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.952 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T00:59:30.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.956 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 105.931699165, 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T00:59:30.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T00:59:30.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 105.93162823199999, 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.961 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T00:59:30.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.965 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 105.931568626, 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T00:59:30.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T00:59:30.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 105.931509256, 2024-09-18T00:59:30.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T00:59:30.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T00:59:30.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 105.931447467, 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.975 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T00:59:30.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 105.931314644, 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T00:59:30.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.376097167, 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.984 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T00:59:30.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:30.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:30.990 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.375597043, 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:30.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T00:59:30.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.375566535, 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:30.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:30.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T00:59:30.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:30.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.37554187000001, 2024-09-18T00:59:30.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:31.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:31.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T00:59:31.003 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.375513338, 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:31.004 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:31.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T00:59:31.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:31.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:31.008 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.375180047, 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:31.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T00:59:31.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.37513819599999, 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:31.013 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T00:59:31.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T00:59:31.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T00:59:31.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.3741007, 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:31.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T00:59:31.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:31.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:31.023 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.721812694999997, 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:31.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T00:59:31.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T00:59:31.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:138250 3.c 3:32730bfd:::rbd_data.117c1c3044f5.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:21.758914+0000", 2024-09-18T00:59:31.029 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.1761148870000007, 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060687680000000001, 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 138250 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758917+0000", 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5550000000000001e-06 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:31.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758925+0000", 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5900000000000008e-06 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758947+0000", 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1197000000000001e-05 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758957+0000", 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0137e-05 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759094+0000", 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013775300000000001 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759246+0000", 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015127900000000001 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.770839+0000", 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011593538 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819413+0000", 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048573660999999997 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819467+0000", 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3795999999999998e-05 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819529+0000", 2024-09-18T00:59:31.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2675999999999994e-05 2024-09-18T00:59:31.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:31.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:31.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:31.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819602+0000", 2024-09-18T00:59:31.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2497999999999996e-05 2024-09-18T00:59:31.035 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.035 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:31.035 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.035 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:31.035 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:31.035 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:31.037 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:31.037+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:31.037 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:31.137 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:31.133+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:31.173 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:31.173 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:31.173 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:31.173 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:31.192 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:31.238 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:31.237+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:31.339 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:31.337+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:31.372 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:31.372 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:31.386 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:31.440 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:31.437+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:31.509 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.441946061, 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:59:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.549412242000002, 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T00:59:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T00:59:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.800345451999998, 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T00:59:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.789970996000001, 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T00:59:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.789898313999998, 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T00:59:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.789867447999999, 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T00:59:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.789814673999999, 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T00:59:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.789788508000001, 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T00:59:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.789763353000001, 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.789340041999999, 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T00:59:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.789301664, 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T00:59:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T00:59:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T00:59:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.789276899000001, 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.782482134999999, 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.568 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T00:59:31.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.782456535000001, 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T00:59:31.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T00:59:31.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.782433191999999, 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T00:59:31.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T00:59:31.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.782410753000001, 2024-09-18T00:59:31.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:31.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T00:59:31.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T00:59:31.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.782389980000001, 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.782368953999999, 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T00:59:31.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T00:59:31.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.782328371999998, 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T00:59:31.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.782294273000002, 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T00:59:31.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T00:59:31.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:31.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T00:59:31.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T00:59:31.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.604 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.604 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.604 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:31.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:31.606 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:31.605+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:31.706 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:31.705+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:31.781 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:31.781 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:31.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.781 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:140635 3.d 3.272a010d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-18T00:59:31.781 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:31.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.441795198, 2024-09-18T00:59:31.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.441840395, 2024-09-18T00:59:31.781 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "waiting for peered", 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 140635 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338625+0000", 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338634+0000", 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9099999999999994e-06 2024-09-18T00:59:31.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338657+0000", 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3672999999999998e-05 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338675+0000", 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7589999999999999e-05 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338767+0000", 2024-09-18T00:59:31.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1896000000000002e-05 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338776+0000", 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.38e-06 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 1 2024-09-18T00:59:31.785 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:31.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:31.838 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:31.837+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:31.908 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:31.909+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:31.963 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:31.963 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:31.963 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:31.963 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:31.963 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:31.977 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:32.022 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:32.021+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:32.101 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 128.80856514000001, 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:59:32.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:59:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.612224087, 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:59:32.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.61220265, 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:32.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:59:32.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:32.116 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.612107323, 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:32.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:59:32.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:59:32.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.61208188400001, 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:59:32.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:59:32.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:59:32.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.61205936100001, 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.612035921, 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:59:32.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.612012111, 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:32.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.138 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.61198974600001, 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:59:32.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.611963129, 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:59:32.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:59:32.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7498 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1998848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.60781135800001, 2024-09-18T00:59:32.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045453341000000001, 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7498 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490074+0000", 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490076+0000", 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3130000000000001e-06 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490082+0000", 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0050000000000001e-06 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490088+0000", 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.057e-06 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535370+0000", 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045282085 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535450+0000", 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0734000000000003e-05 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535527+0000", 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6747000000000002e-05 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.607790615, 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.152 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:59:32.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.60774505000001, 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.156 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:32.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:59:32.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:32.160 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.607718709, 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:32.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.607696039, 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:59:32.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.607677666, 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.173 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.607657878, 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.607631008, 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:59:32.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.531440539000002, 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:32.187 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.120375443, 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T00:59:32.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:32.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.194 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.194 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:32.194 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6] out_osds: [2, 3, 0, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:59:32.194 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:59:32.194 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T00:59:32.195 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T00:59:32.195 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:32.193+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:32.196 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:32.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:59:32.281 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone38 from 0xe681037 (overlap 0x3c985d) to 0x183cf2 2024-09-18T00:59:32.326 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:32.325+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:32.372 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:32.372 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:32.373 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:32.373 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:32.390 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:32.434 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:32.433+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:32.498 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:32.497+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:32.555 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:32.556 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:32.570 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 119, 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "82a96d76-ba83-49e2-a985-84604bd599ba", 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-18T00:57:07.138732+0000", 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-18T00:59:32.180495+0000", 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-18T00:57:20.258605+0000", 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-18T00:59:27.145670+0000", 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5804032, 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-18T00:59:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "luminous", 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-18T00:59:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:21.962521+0000", 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "19", 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-18T00:59:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 8, 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 8, 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.25, 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:25.721733+0000", 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "23", 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 23, 2024-09-18T00:59:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 3, 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 3, 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.5, 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.5, 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.5, 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:35.599413+0000", 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "117", 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 41, 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 117, 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T00:59:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 3, 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 3, 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.5, 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.5, 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.5, 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "3f359ce5-6f6e-4161-8554-8ac028347274", 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 95, 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/199763443", 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/199763443", 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/199763443", 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/199763443", 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:59:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "48687611-1556-4b6d-86af-7872494a91d7", 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 118, 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/345149800", 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/345149800", 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/345149800", 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/345149800", 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "9e6b1f90-ce9b-46d4-92fe-d457f7e415d5", 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/431239931", 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/431239931", 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/431239931", 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/431239931", 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5", 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 89, 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/1775474180", 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/1775474180", 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/1775474180", 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/1775474180", 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "802ccbeb-7351-4b38-832a-695117c7e493", 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 117, 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6808", 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6809", 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6810", 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6811", 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6814", 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6815", 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6812", 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6813", 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6809/4198401207", 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6811/4198401207", 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6815/4198401207", 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6813/4198401207", 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d958b3da-efa1-4d86-b81d-e78874441437", 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 117, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6824", 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6825", 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6826", 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6827", 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6830", 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6831", 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6828", 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6829", 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6825/2599825823", 2024-09-18T00:59:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6827/2599825823", 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6831/2599825823", 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6829/2599825823", 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "51b3cea8-c686-4fd3-b3f4-ba936cf26b64", 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 117, 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6800", 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6801", 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6802", 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6803", 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6806", 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6807", 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6804", 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6805", 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6801/3550656451", 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6803/3550656451", 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6807/3550656451", 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6805/3550656451", 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b02ead1c-837a-4b07-9ad1-4e903ed5bc4e", 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 106, 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6816", 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6817", 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6818", 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6819", 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6822", 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6823", 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6820", 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6821", 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6817/2993352359", 2024-09-18T00:59:32.639 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6819/2993352359", 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6823/2993352359", 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6821/2993352359", 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:32.640 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.896477+0000", 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.941756+0000", 2024-09-18T00:59:32.641 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.560180+0000", 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:59:32.642 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.077394+0000", 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.996264+0000", 2024-09-18T00:59:32.643 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.511686+0000", 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:59:32.644 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.411051+0000", 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:32.645 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.888092+0000", 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [ 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "2.1", 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: 5, 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: 3 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:32.646 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.647 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [], 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.648 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:32.649 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:32.651 INFO:tasks.thrashosds.thrasher:j is {'epoch': 119, 'fsid': '82a96d76-ba83-49e2-a985-84604bd599ba', 'created': '2024-09-18T00:57:07.138732+0000', 'modified': '2024-09-18T00:59:32.180495+0000', 'last_up_change': '2024-09-18T00:57:20.258605+0000', 'last_in_change': '2024-09-18T00:59:27.145670+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T00:57:21.962521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T00:57:25.721733+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.5, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T00:57:35.599413+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 41, 'snap_epoch': 117, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.5, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3f359ce5-6f6e-4161-8554-8ac028347274', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 199763443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 199763443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 199763443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 199763443}]}, 'public_addr': '172.21.15.42:6801/199763443', 'cluster_addr': '172.21.15.42:6803/199763443', 'heartbeat_back_addr': '172.21.15.42:6807/199763443', 'heartbeat_front_addr': '172.21.15.42:6805/199763443', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '48687611-1556-4b6d-86af-7872494a91d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 345149800}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 345149800}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 345149800}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 345149800}]}, 'public_addr': '172.21.15.42:6809/345149800', 'cluster_addr': '172.21.15.42:6811/345149800', 'heartbeat_back_addr': '172.21.15.42:6815/345149800', 'heartbeat_front_addr': '172.21.15.42:6813/345149800', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9e6b1f90-ce9b-46d4-92fe-d457f7e415d5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 431239931}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 431239931}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 431239931}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 431239931}]}, 'public_addr': '172.21.15.42:6825/431239931', 'cluster_addr': '172.21.15.42:6827/431239931', 'heartbeat_back_addr': '172.21.15.42:6831/431239931', 'heartbeat_front_addr': '172.21.15.42:6829/431239931', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 89, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 1775474180}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 1775474180}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 1775474180}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 1775474180}]}, 'public_addr': '172.21.15.42:6817/1775474180', 'cluster_addr': '172.21.15.42:6819/1775474180', 'heartbeat_back_addr': '172.21.15.42:6823/1775474180', 'heartbeat_front_addr': '172.21.15.42:6821/1775474180', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '802ccbeb-7351-4b38-832a-695117c7e493', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6809', 'nonce': 4198401207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6811', 'nonce': 4198401207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6815', 'nonce': 4198401207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6813', 'nonce': 4198401207}]}, 'public_addr': '172.21.15.195:6809/4198401207', 'cluster_addr': '172.21.15.195:6811/4198401207', 'heartbeat_back_addr': '172.21.15.195:6815/4198401207', 'heartbeat_front_addr': '172.21.15.195:6813/4198401207', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd958b3da-efa1-4d86-b81d-e78874441437', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6824', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6825', 'nonce': 2599825823}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6826', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6827', 'nonce': 2599825823}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6830', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6831', 'nonce': 2599825823}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6828', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6829', 'nonce': 2599825823}]}, 'public_addr': '172.21.15.195:6825/2599825823', 'cluster_addr': '172.21.15.195:6827/2599825823', 'heartbeat_back_addr': '172.21.15.195:6831/2599825823', 'heartbeat_front_addr': '172.21.15.195:6829/2599825823', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '51b3cea8-c686-4fd3-b3f4-ba936cf26b64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6800', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6801', 'nonce': 3550656451}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6803', 'nonce': 3550656451}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6807', 'nonce': 3550656451}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6805', 'nonce': 3550656451}]}, 'public_addr': '172.21.15.195:6801/3550656451', 'cluster_addr': '172.21.15.195:6803/3550656451', 'heartbeat_back_addr': '172.21.15.195:6807/3550656451', 'heartbeat_front_addr': '172.21.15.195:6805/3550656451', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b02ead1c-837a-4b07-9ad1-4e903ed5bc4e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6817', 'nonce': 2993352359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6818', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6819', 'nonce': 2993352359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6822', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6823', 'nonce': 2993352359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6820', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6821', 'nonce': 2993352359}]}, 'public_addr': '172.21.15.195:6817/2993352359', 'cluster_addr': '172.21.15.195:6819/2993352359', 'heartbeat_back_addr': '172.21.15.195:6823/2993352359', 'heartbeat_front_addr': '172.21.15.195:6821/2993352359', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.896477+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.941756+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.560180+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.077394+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.996264+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.511686+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.411051+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.888092+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.1', 'osds': [5, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T00:59:32.651 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 2.1 2024-09-18T00:59:32.651 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 2.1 2024-09-18T00:59:32.651 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-18T00:59:32.652 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:32.649+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:32.752 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:32.749+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.710628785, 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:59:32.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:59:32.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.709699194, 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:32.799 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.709685134, 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.708394559, 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.804 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:59:32.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:59:32.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:59:32.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.708374087, 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:59:32.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:59:32.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.705579634, 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.70556445, 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:59:32.816 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:59:32.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:59:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.705541504, 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:59:32.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:59:32.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:59:32.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.701363865, 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:59:32.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:59:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.70134264799999, 2024-09-18T00:59:32.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.833 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.701321265, 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:32.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:59:32.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.69742141099999, 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.838 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:59:32.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:59:32.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.69740720999999, 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:59:32.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.846 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.697389173, 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.697375107, 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.851 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.693349756, 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.859 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.693335744, 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.69332178000001, 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:59:32.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:59:32.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.931463826, 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.872 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 85.805987454999993, 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:59:32.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:59:32.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:32.878 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:32.880 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:32.881 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:32.882 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:32.882 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:32.882 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:32.883 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:32.883 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:32.881+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:32.883 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:32.884 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:32.900 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:32.900 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:32.921 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:32.942 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:32.942 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:32.968 INFO:teuthology.orchestra.run.smithi042.stdout:5118 punch from 0x564c1e4 to 0x56567b6, (0xa5d2 bytes) 2024-09-18T00:59:32.974 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:32.974 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:32.978 INFO:teuthology.orchestra.run.smithi042.stdout:5119 punch from 0x1209fce to 0x1215772, (0xb7a4 bytes) 2024-09-18T00:59:32.978 INFO:teuthology.orchestra.run.smithi042.stdout:5120 trunc from 0xe681037 to 0xe420395 2024-09-18T00:59:32.989 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:32.989+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:33.000 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:33.000 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:33.012 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:33.012 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:33.016 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:33.085 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:33.085+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:33.113 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:33.113 INFO:teuthology.orchestra.run.smithi042.stdout:5122 write 0x16fb754 thru 0x170a08a (0xe937 bytes) 2024-09-18T00:59:33.115 INFO:teuthology.orchestra.run.smithi042.stdout:5123 write 0xa81f881 thru 0xa827059 (0x77d9 bytes) 2024-09-18T00:59:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:5125 write 0x600a269 thru 0x600e4f0 (0x4288 bytes) 2024-09-18T00:59:33.119 INFO:teuthology.orchestra.run.smithi042.stdout:5126 read 0x14ab0d0 thru 0x14b5ab6 (0xa9e7 bytes) 2024-09-18T00:59:33.125 INFO:teuthology.orchestra.run.smithi042.stdout:5127 trunc from 0xe420395 to 0x9813afb 2024-09-18T00:59:33.137 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:33.137 INFO:teuthology.orchestra.run.smithi042.stdout:5129 read 0x66e7ebd thru 0x66ead89 (0x2ecd bytes) 2024-09-18T00:59:33.137 INFO:teuthology.orchestra.run.smithi042.stdout:5130 write 0x67419cf thru 0x675196c (0xff9e bytes) 2024-09-18T00:59:33.140 INFO:teuthology.orchestra.run.smithi042.stdout:5132 read 0x10a6d29 thru 0x10b3618 (0xc8f0 bytes) 2024-09-18T00:59:33.141 INFO:teuthology.orchestra.run.smithi042.stdout:5133 read 0x5c99f13 thru 0x5ca6ca2 (0xcd90 bytes) 2024-09-18T00:59:33.156 INFO:teuthology.orchestra.run.smithi042.stdout:5135 write 0xe45387b thru 0xe462976 (0xf0fc bytes) 2024-09-18T00:59:33.158 INFO:teuthology.orchestra.run.smithi042.stdout:5136 write 0x69bea27 thru 0x69c0e60 (0x243a bytes) 2024-09-18T00:59:33.169 INFO:teuthology.orchestra.run.smithi042.stdout:5137 write 0xeda5b18 thru 0xedb4017 (0xe500 bytes) 2024-09-18T00:59:33.171 INFO:teuthology.orchestra.run.smithi042.stdout:5138 punch from 0x83538f0 to 0x835d1ac, (0x98bc bytes) 2024-09-18T00:59:33.171 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:33.176 INFO:teuthology.orchestra.run.smithi042.stdout:5140 trunc from 0xedb4018 to 0xb2565a6 2024-09-18T00:59:33.186 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:33.185+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:33.195 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:33.195 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:59:33.196 INFO:teuthology.orchestra.run.smithi042.stdout:5142 trunc from 0xb2565a6 to 0x7712a26 2024-09-18T00:59:33.196 INFO:teuthology.orchestra.run.smithi042.stderr:clear 2.1 pg_upmap mapping 2024-09-18T00:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout:5145 punch from 0x5fd2b94 to 0x5fdd5f7, (0xaa63 bytes) 2024-09-18T00:59:33.213 INFO:teuthology.orchestra.run.smithi042.stdout:5147 writesame 0x870e29 thru 0x877746 data_size 0x45(0x691e bytes) 2024-09-18T00:59:33.214 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:33.215 INFO:teuthology.orchestra.run.smithi042.stdout:5150 read 0x66ebc6d thru 0x66f765f (0xb9f3 bytes) 2024-09-18T00:59:33.217 INFO:teuthology.orchestra.run.smithi042.stdout:5152 write 0xf19fdf thru 0xf1c034 (0x2056 bytes) 2024-09-18T00:59:33.218 INFO:teuthology.orchestra.run.smithi042.stdout:5153 write 0x2a4e7ae thru 0x2a5ad10 (0xc563 bytes) 2024-09-18T00:59:33.232 INFO:teuthology.orchestra.run.smithi042.stdout:5155 write 0xa41b762 thru 0xa422ee0 (0x777f bytes) 2024-09-18T00:59:33.247 INFO:teuthology.orchestra.run.smithi042.stdout:5156 writesame 0xc2ec400 thru 0xc2f3f77 data_size 0x18(0x7b78 bytes) 2024-09-18T00:59:33.249 INFO:teuthology.orchestra.run.smithi042.stdout:5157 write 0x5076227 thru 0x507aa02 (0x47dc bytes) 2024-09-18T00:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:33.252 INFO:teuthology.orchestra.run.smithi042.stdout:5159 writesame 0x8c1b4b7 thru 0x8c24eae data_size 0x34(0x99f8 bytes) 2024-09-18T00:59:33.258 INFO:teuthology.orchestra.run.smithi042.stdout:5160 trunc from 0xc2f3f78 to 0xe994807 2024-09-18T00:59:33.270 INFO:teuthology.orchestra.run.smithi042.stdout:5161 read 0x9cf8df3 thru 0x9d05dbb (0xcfc9 bytes) 2024-09-18T00:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout:5162 punch from 0x8626aa2 to 0x8626ad8, (0x36 bytes) 2024-09-18T00:59:33.271 INFO:teuthology.orchestra.run.smithi042.stdout:5163 write 0xcc1744d thru 0xcc19862 (0x2416 bytes) 2024-09-18T00:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout:5164 punch from 0xae6ef25 to 0xae72c95, (0x3d70 bytes) 2024-09-18T00:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:33.274 INFO:teuthology.orchestra.run.smithi042.stdout:5168 write 0x6c7a31b thru 0x6c80188 (0x5e6e bytes) 2024-09-18T00:59:33.276 INFO:teuthology.orchestra.run.smithi042.stdout:5169 write 0x52e8fca thru 0x52f5810 (0xc847 bytes) 2024-09-18T00:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout:5170 read 0x3367e5c thru 0x3369036 (0x11db bytes) 2024-09-18T00:59:33.279 INFO:teuthology.orchestra.run.smithi042.stdout:5171 read 0x803a114 thru 0x8044e69 (0xad56 bytes) 2024-09-18T00:59:33.280 INFO:teuthology.orchestra.run.smithi042.stdout:5172 write 0x91f6a95 thru 0x91fc5cf (0x5b3b bytes) 2024-09-18T00:59:33.282 INFO:teuthology.orchestra.run.smithi042.stdout:5173 writesame 0xa390edb thru 0xa39e99f data_size 0x1(0xdac5 bytes) 2024-09-18T00:59:33.289 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:33.285+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:33.328 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:33.333 INFO:teuthology.orchestra.run.smithi042.stdout:5174 writesame 0xbef8d7 thru 0xbf35ef data_size 0x1(0x3d19 bytes) 2024-09-18T00:59:33.342 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:33.362 INFO:teuthology.orchestra.run.smithi042.stdout:5175 write 0xee378c4 thru 0xee3a58d (0x2cca bytes) 2024-09-18T00:59:33.366 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:33.366 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:33.366 INFO:teuthology.orchestra.run.smithi042.stdout:5182 clone 40 order 22 su 16384 sc 13 2024-09-18T00:59:33.390 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:33.389+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:33.466 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:33.466 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:33.480 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:33.521 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:33.521+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:33.592 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:33.589+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:33.644 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:33.644 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:33.659 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:33.692 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:33.689+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:33.782 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:33.782 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:33.793 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:33.789+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:33.796 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:33.894 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:33.893+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:33.919 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:33.920 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:33.934 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:34.025 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:34.025+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:34.095 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:34.093+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:34.098 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:34.098 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:34.112 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:34.196 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:34.197+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:34.282 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:34.282 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:34.296 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:34.297 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:34.293+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:34.398 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:34.397+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:34.466 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:34.466 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:34.480 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:34.524 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:34.525+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:34.600 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:34.597+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:34.603 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:34.603 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:34.617 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:34.701 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:34.697+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:34.704 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone39 2024-09-18T00:59:34.783 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:34.784 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:34.798 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:34.802 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:34.801+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:34.903 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:34.901+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:34.925 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:34.925 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:34.941 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:35.003 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:35.001+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:35.030 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:35.087 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-18T00:59:35.104 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:35.101+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:35.109 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:35.110 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:35.126 INFO:teuthology.orchestra.run.smithi042.stdout:5183 punch from 0x1a1eb01 to 0x1a20052, (0x1551 bytes) 2024-09-18T00:59:35.126 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:35.133 INFO:teuthology.orchestra.run.smithi042.stdout:5184 writesame 0x19692a4 thru 0x196eb52 data_size 0x1(0x58af bytes) 2024-09-18T00:59:35.192 INFO:teuthology.orchestra.run.smithi042.stdout:5186 punch from 0x6f53c37 to 0x6f566a5, (0x2a6e bytes) 2024-09-18T00:59:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:5187 read 0x4cce66b thru 0x4cd0358 (0x1cee bytes) 2024-09-18T00:59:35.196 INFO:teuthology.orchestra.run.smithi042.stdout:5190 read 0x9de0d88 thru 0x9dec35d (0xb5d6 bytes) 2024-09-18T00:59:35.205 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:35.201+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:35.207 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:35.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:59:35.236 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:35.236 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:35.236 INFO:teuthology.orchestra.run.smithi042.stdout:5192 trunc from 0xee3a58e to 0x5ff4d8d 2024-09-18T00:59:35.255 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:35.255 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:35.287 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:35.288 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:35.297 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:35.297 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:35.305 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:35.305+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:35.308 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:35.309 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:35.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:35.336 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:35.337 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:35.375 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:35.413 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:35.409+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:35.418 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:35.418 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:35.497 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:35.497 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:35.498 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:35.508 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:35.505+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:35.513 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:35.514 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:35.530 INFO:teuthology.orchestra.run.smithi042.stdout:5194 write 0xc27273e thru 0xc27452b (0x1dee bytes) 2024-09-18T00:59:35.534 INFO:teuthology.orchestra.run.smithi042.stdout:5197 punch from 0x1d538b8 to 0x1d5f75c, (0xbea4 bytes) 2024-09-18T00:59:35.542 INFO:teuthology.orchestra.run.smithi042.stdout:5198 write 0xd0d3d14 thru 0xd0e1d03 (0xdff0 bytes) 2024-09-18T00:59:35.552 INFO:teuthology.orchestra.run.smithi042.stdout:5199 writesame 0x5900500 thru 0x590438c data_size 0x1(0x3e8d bytes) 2024-09-18T00:59:35.564 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:35.564 INFO:teuthology.orchestra.run.smithi042.stdout:5201 clone 41 order 19 su 262144 sc 12 2024-09-18T00:59:35.608 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:35.605+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:35.633 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:35.633 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:35.633 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:35.633 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:35.634 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:35.648 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:35.710 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:35.709+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:35.811 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:35.809+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:35.819 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:35.819 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:35.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 106.751746904, 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:59:35.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:59:35.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.859213085, 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.110146295, 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T00:59:35.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.835 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.099771838999999, 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T00:59:35.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T00:59:35.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.099699157, 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:35.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.845 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.845 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:35.845 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.099668291, 2024-09-18T00:59:35.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T00:59:35.845 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T00:59:35.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.099615517, 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.850 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T00:59:35.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T00:59:35.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.099589350999999, 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T00:59:35.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.858 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.099564195999999, 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T00:59:35.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T00:59:35.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T00:59:35.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T00:59:35.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.863 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.099140885000001, 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.864 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T00:59:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T00:59:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T00:59:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T00:59:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T00:59:35.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T00:59:35.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T00:59:35.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T00:59:35.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T00:59:35.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.868 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.099102507000001, 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T00:59:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T00:59:35.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.873 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.099077741999999, 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:59:35.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.092282978, 2024-09-18T00:59:35.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T00:59:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T00:59:35.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:35.883 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.092257377999999, 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T00:59:35.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.092234035000001, 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T00:59:35.888 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:35.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.092211595999999, 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T00:59:35.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T00:59:35.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.092190822999999, 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T00:59:35.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.092169797, 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T00:59:35.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T00:59:35.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.092129215, 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:35.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T00:59:35.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T00:59:35.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T00:59:35.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T00:59:35.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T00:59:35.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.911 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.092095115999999, 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:35.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:35.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:35.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:35.916 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:35.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:35.918 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:35.917+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:36.019 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:36.017+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:36.094 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:36.095 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:36.095 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:36.095 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:36.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:36.150 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:36.149+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:36.214 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:59:36.221 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:36.221+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:36.282 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:36.297 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:36.322 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:36.321+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:36.329 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone40 from 0xd0e1d04 (overlap 0x5ff4d8d) to 0x432b649 2024-09-18T00:59:36.423 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:36.421+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:36.425 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 133.133823643, 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:36.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:59:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:59:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:59:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.93748259, 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:59:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:59:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.937461153, 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:59:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.937365826, 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:59:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.93734038700001, 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:59:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.93731786399999, 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.937294424, 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:59:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:59:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:59:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.937270614, 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:59:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.93724824900001, 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:59:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.937221632, 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:59:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:59:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.933049118, 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:59:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.93300355300001, 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:59:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:59:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.932977212, 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:59:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.932954542, 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:59:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.932936169, 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:59:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.932916381, 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:59:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:59:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:59:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.932889511, 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:59:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.856699042000002, 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T00:59:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.445633946000001, 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T00:59:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:140635 3.d 3:b08054e4:::rbd_object_map.117ced55205b.0000000000000023:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-18T00:59:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.0845205519999999, 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86222047999999996, 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 140635 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338625+0000", 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338634+0000", 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9099999999999994e-06 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338657+0000", 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3672999999999998e-05 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338675+0000", 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7589999999999999e-05 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338767+0000", 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1896000000000002e-05 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338776+0000", 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.38e-06 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200213+0000", 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86143668200000001 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200432+0000", 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021884199999999999 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200844+0000", 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000411887 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:36.520 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:36.522 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:36.523 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:36.521+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:36.572 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-18T00:59:36.624 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:36.625+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:36.701 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:36.718 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:36.765 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:36.761+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:36.826 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:36.825+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:36.888 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:36.888 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:36.889 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:36.889 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:36.889 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:36.906 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:36.954 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:36.953+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:37.028 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:37.025+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:37.038 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:37.038 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:37.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:37.038 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:37.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.038 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.957699347, 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:59:37.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:59:37.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.956769756, 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.047 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.956755696, 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.955465121, 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.052 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:59:37.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:59:37.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.955444649, 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:59:37.056 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:59:37.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:59:37.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.95265019599999, 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:59:37.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.064 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.952635012, 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.065 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:59:37.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:59:37.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:59:37.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.952612066, 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:59:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.073 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.948434427, 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.074 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:59:37.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:59:37.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:59:37.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.94841321, 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:59:37.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.94839182699999, 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:59:37.082 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:59:37.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.944491973, 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:59:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:37.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.091 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.944477772, 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:59:37.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:59:37.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.944459735, 2024-09-18T00:59:37.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.100 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.944445669, 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:59:37.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:59:37.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.94042031799999, 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.108 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.940406306, 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.940392342, 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:37.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:59:37.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.178534388, 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:59:37.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.053058016999998, 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:59:37.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:59:37.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:59:37.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.127 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:37.128 INFO:teuthology.orchestra.run.smithi042.stdout:5202 write 0x883e800 thru 0x884806a (0x986b bytes) 2024-09-18T00:59:37.129 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:37.130 INFO:teuthology.orchestra.run.smithi042.stdout:5203 read 0xc65404 thru 0xc6ecdf (0x98dc bytes) 2024-09-18T00:59:37.130 INFO:teuthology.orchestra.run.smithi042.stdout:5204 read 0x279bcb9 thru 0x27aa5f0 (0xe938 bytes) 2024-09-18T00:59:37.130 INFO:teuthology.orchestra.run.smithi042.stdout:5208 write 0xb1a3b9c thru 0xb1a703c (0x34a1 bytes) 2024-09-18T00:59:37.130 INFO:teuthology.orchestra.run.smithi042.stdout:5209 write 0x6e67de6 thru 0x6e7161d (0x9838 bytes) 2024-09-18T00:59:37.130 INFO:teuthology.orchestra.run.smithi042.stdout:5210 writesame 0xd8973e4 thru 0xd89ba88 data_size 0x1(0x46a5 bytes) 2024-09-18T00:59:37.130 INFO:teuthology.orchestra.run.smithi042.stdout:5211 write 0x4a2bcd3 thru 0x4a324a0 (0x67ce bytes) 2024-09-18T00:59:37.130 INFO:teuthology.orchestra.run.smithi042.stdout:5212 read 0xc9e9ecf thru 0xc9f9a76 (0xfba8 bytes) 2024-09-18T00:59:37.131 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:37.125+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:37.152 INFO:teuthology.orchestra.run.smithi042.stdout:5213 writesame 0xde6fb05 thru 0xde7d7c1 data_size 0x1(0xdcbd bytes) 2024-09-18T00:59:37.152 INFO:teuthology.orchestra.run.smithi042.stdout:5214 write 0x7f4454a thru 0x7f5378e (0xf245 bytes) 2024-09-18T00:59:37.160 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:37.160 INFO:teuthology.orchestra.run.smithi042.stdout:5216 read 0x52cd597 thru 0x52d425f (0x6cc9 bytes) 2024-09-18T00:59:37.162 INFO:teuthology.orchestra.run.smithi042.stdout:5218 writesame 0x84c6a6b thru 0x84cfbd8 data_size 0x49(0x916e bytes) 2024-09-18T00:59:37.176 INFO:teuthology.orchestra.run.smithi042.stdout:5220 writesame 0x1000a36 thru 0x1002f8e data_size 0x1(0x2559 bytes) 2024-09-18T00:59:37.219 INFO:teuthology.orchestra.run.smithi042.stdout:5221 trunc from 0xde7d7c2 to 0x1fbb24 2024-09-18T00:59:37.230 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:37.229+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout:5222 punch from 0x19b766 to 0x1a3620, (0x7eba bytes) 2024-09-18T00:59:37.296 INFO:teuthology.orchestra.run.smithi042.stdout:5224 read 0x1ad123 thru 0x1bac75 (0xdb53 bytes) 2024-09-18T00:59:37.301 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:37.301 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:37.314 INFO:teuthology.orchestra.run.smithi042.stdout:5225 writesame 0x2d98d35 thru 0x2da07b6 data_size 0x3(0x7a82 bytes) 2024-09-18T00:59:37.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:37.330 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:37.329+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:37.341 INFO:teuthology.orchestra.run.smithi042.stdout:5228 write 0xa434a36 thru 0xa441b74 (0xd13f bytes) 2024-09-18T00:59:37.349 INFO:teuthology.orchestra.run.smithi042.stdout:5229 write 0x863f0d3 thru 0x86458e7 (0x6815 bytes) 2024-09-18T00:59:37.366 INFO:teuthology.orchestra.run.smithi042.stdout:5230 trunc from 0xa441b75 to 0xe25a740 2024-09-18T00:59:37.376 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:37.377 INFO:teuthology.orchestra.run.smithi042.stdout:5233 write 0x6c5f928 thru 0x6c5ffc9 (0x6a2 bytes) 2024-09-18T00:59:37.383 INFO:teuthology.orchestra.run.smithi042.stdout:5234 write 0xd1388bd thru 0xd13c8b7 (0x3ffb bytes) 2024-09-18T00:59:37.388 INFO:teuthology.orchestra.run.smithi042.stdout:5235 write 0xbe3ad0d thru 0xbe3d9bc (0x2cb0 bytes) 2024-09-18T00:59:37.394 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:37.394 INFO:teuthology.orchestra.run.smithi042.stdout:5237 write 0x6049414 thru 0x6051145 (0x7d32 bytes) 2024-09-18T00:59:37.401 INFO:teuthology.orchestra.run.smithi042.stdout:5238 read 0xa3dcf23 thru 0xa3e905f (0xc13d bytes) 2024-09-18T00:59:37.401 INFO:teuthology.orchestra.run.smithi042.stdout:5239 writesame 0x9eedc7 thru 0x9fd568 data_size 0x3(0xe7a2 bytes) 2024-09-18T00:59:37.428 INFO:teuthology.orchestra.run.smithi042.stdout:5240 punch from 0x58b260e to 0x58bc531, (0x9f23 bytes) 2024-09-18T00:59:37.428 INFO:teuthology.orchestra.run.smithi042.stdout:5241 writesame 0x50d7ed8 thru 0x50e261f data_size 0x8(0xa748 bytes) 2024-09-18T00:59:37.431 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:37.429+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:37.441 INFO:teuthology.orchestra.run.smithi042.stdout:5242 write 0xc6579a5 thru 0xc665545 (0xdba1 bytes) 2024-09-18T00:59:37.445 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:37.448 INFO:teuthology.orchestra.run.smithi042.stdout:5243 punch from 0x4bbc668 to 0x4bc46f7, (0x808f bytes) 2024-09-18T00:59:37.458 INFO:teuthology.orchestra.run.smithi042.stdout:5244 trunc from 0xe25a740 to 0x86b92e0 2024-09-18T00:59:37.473 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:37.473 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:37.474 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:37.474 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:37.474 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:37.484 INFO:teuthology.orchestra.run.smithi042.stdout:5245 write 0x867cfb4 thru 0x8687d57 (0xada4 bytes) 2024-09-18T00:59:37.490 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:5246 read 0x1366cd4 thru 0x1374172 (0xd49f bytes) 2024-09-18T00:59:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:5247 write 0x4bf9ba2 thru 0x4c08b04 (0xef63 bytes) 2024-09-18T00:59:37.500 INFO:teuthology.orchestra.run.smithi042.stdout:5249 writesame 0x424671c thru 0x4255c20 data_size 0x41(0xf505 bytes) 2024-09-18T00:59:37.513 INFO:teuthology.orchestra.run.smithi042.stdout:5250 trunc from 0x86b92e0 to 0xbc8fb97 2024-09-18T00:59:37.522 INFO:teuthology.orchestra.run.smithi042.stdout:5251 write 0x806bb55 thru 0x806d021 (0x14cd bytes) 2024-09-18T00:59:37.529 INFO:teuthology.orchestra.run.smithi042.stdout:5254 punch from 0x6a8c821 to 0x6a9693a, (0xa119 bytes) 2024-09-18T00:59:37.529 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:37.529 INFO:teuthology.orchestra.run.smithi042.stdout:5256 read 0xa759a18 thru 0xa75ade6 (0x13cf bytes) 2024-09-18T00:59:37.529 INFO:teuthology.orchestra.run.smithi042.stdout:5257 write 0x5c28708 thru 0x5c2f80d (0x7106 bytes) 2024-09-18T00:59:37.532 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:37.529+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:37.550 INFO:teuthology.orchestra.run.smithi042.stdout:5258 write 0xec3838e thru 0xec3e8f8 (0x656b bytes) 2024-09-18T00:59:37.557 INFO:teuthology.orchestra.run.smithi042.stdout:5261 writesame 0x6672c04 thru 0x66750f1 data_size 0x1d(0x24ee bytes) 2024-09-18T00:59:37.564 INFO:teuthology.orchestra.run.smithi042.stdout:5262 read 0xb545d97 thru 0xb554303 (0xe56d bytes) 2024-09-18T00:59:37.564 INFO:teuthology.orchestra.run.smithi042.stdout:5263 read 0x3162500 thru 0x31694a7 (0x6fa8 bytes) 2024-09-18T00:59:37.565 INFO:teuthology.orchestra.run.smithi042.stdout:5264 writesame 0x23b8978 thru 0x23c54ca data_size 0x1(0xcb53 bytes) 2024-09-18T00:59:37.604 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:37.604 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:37.622 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:37.622 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:37.623 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:37.623 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:37.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:37.623 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:37.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.623 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98052 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4179968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.623 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:37.623 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.181234429, 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131360583, 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98052 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:37.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:37.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440037+0000", 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440040+0000", 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7159999999999999e-06 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440048+0000", 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7169999999999997e-06 2024-09-18T00:59:37.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440056+0000", 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1860000000000003e-06 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.570989+0000", 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13093384999999999 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571175+0000", 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018504799999999999 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571397+0000", 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022245899999999999 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.628 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98053 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4466688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.18120656, 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13182195299999999, 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98053 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.629 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440065+0000", 2024-09-18T00:59:37.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:37.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440068+0000", 2024-09-18T00:59:37.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6699999999999998e-06 2024-09-18T00:59:37.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440076+0000", 2024-09-18T00:59:37.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7069999999999998e-06 2024-09-18T00:59:37.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440083+0000", 2024-09-18T00:59:37.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9929999999999998e-06 2024-09-18T00:59:37.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571475+0000", 2024-09-18T00:59:37.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13139158100000001 2024-09-18T00:59:37.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571657+0000", 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018247099999999999 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571886+0000", 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022892400000000001 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98054 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2113536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.181178699999997, 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13228911600000001, 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98054 2024-09-18T00:59:37.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440093+0000", 2024-09-18T00:59:37.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:37.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440096+0000", 2024-09-18T00:59:37.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:37.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440105+0000", 2024-09-18T00:59:37.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7739999999999993e-06 2024-09-18T00:59:37.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440112+0000", 2024-09-18T00:59:37.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2130000000000002e-06 2024-09-18T00:59:37.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571970+0000", 2024-09-18T00:59:37.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13185812199999999 2024-09-18T00:59:37.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572159+0000", 2024-09-18T00:59:37.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000188665 2024-09-18T00:59:37.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572381+0000", 2024-09-18T00:59:37.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022273200000000001 2024-09-18T00:59:37.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.640 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.640 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98055 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2400256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.640 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:37.641 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.181148520000001, 2024-09-18T00:59:37.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273966600000001, 2024-09-18T00:59:37.641 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98055 2024-09-18T00:59:37.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.642 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:37.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:37.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440123+0000", 2024-09-18T00:59:37.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-18T00:59:37.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440126+0000", 2024-09-18T00:59:37.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:37.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440137+0000", 2024-09-18T00:59:37.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0543000000000001e-05 2024-09-18T00:59:37.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440144+0000", 2024-09-18T00:59:37.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-18T00:59:37.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572457+0000", 2024-09-18T00:59:37.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132312807 2024-09-18T00:59:37.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572636+0000", 2024-09-18T00:59:37.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882299999999999 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572862+0000", 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022645200000000001 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98056 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2686976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:37.646 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.181117942999997, 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13308801200000001, 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98056 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440154+0000", 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5300000000000004e-07 2024-09-18T00:59:37.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440156+0000", 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.903e-06 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440165+0000", 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7600000000000008e-06 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440174+0000", 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6969999999999999e-06 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572960+0000", 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132785984 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573087+0000", 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012728699999999999 2024-09-18T00:59:37.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573241+0000", 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153728 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98057 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2973696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.181088350000003, 2024-09-18T00:59:37.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13357376900000001, 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98057 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440183+0000", 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440186+0000", 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8700000000000001e-06 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440195+0000", 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7099999999999996e-06 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440202+0000", 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2429999999999999e-06 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573338+0000", 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13313578400000001 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573526+0000", 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018780399999999999 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573756+0000", 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230755 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98058 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.181060467000002, 2024-09-18T00:59:37.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13387749400000001, 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98058 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:37.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:37.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440627+0000", 2024-09-18T00:59:37.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041660699999999999 2024-09-18T00:59:37.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440634+0000", 2024-09-18T00:59:37.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.596e-06 2024-09-18T00:59:37.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440650+0000", 2024-09-18T00:59:37.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6164e-05 2024-09-18T00:59:37.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440660+0000", 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.023e-05 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573833+0000", 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13317317000000001 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573945+0000", 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111731 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574088+0000", 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000142996 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.662 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98059 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3547136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.662 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:37.662 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.180593119999997, 2024-09-18T00:59:37.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13391484000000001, 2024-09-18T00:59:37.662 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.662 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98059 2024-09-18T00:59:37.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.663 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:37.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:37.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441760+0000", 2024-09-18T00:59:37.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001082185 2024-09-18T00:59:37.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441780+0000", 2024-09-18T00:59:37.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0463e-05 2024-09-18T00:59:37.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441829+0000", 2024-09-18T00:59:37.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8763000000000001e-05 2024-09-18T00:59:37.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441849+0000", 2024-09-18T00:59:37.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9423e-05 2024-09-18T00:59:37.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574173+0000", 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13232460400000001 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574367+0000", 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019403000000000001 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574593+0000", 2024-09-18T00:59:37.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000225372 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98060 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3833856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.179391039000002, 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13320725899999999, 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98060 2024-09-18T00:59:37.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.668 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:37.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441881+0000", 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6160000000000001e-06 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441889+0000", 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8439999999999998e-06 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441909+0000", 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9856000000000001e-05 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441925+0000", 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6039999999999999e-05 2024-09-18T00:59:37.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574669+0000", 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13274412799999999 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574860+0000", 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019038599999999999 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575087+0000", 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022738899999999999 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98061 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4120576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.179324283, 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13362664099999999, 2024-09-18T00:59:37.672 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.673 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.673 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.673 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.673 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98061 2024-09-18T00:59:37.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.673 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:37.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:37.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441948+0000", 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441954+0000", 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3099999999999997e-06 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441981+0000", 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7095999999999999e-05 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441998+0000", 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6237000000000001e-05 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575165+0000", 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13316721200000001 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575355+0000", 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000190218 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575573+0000", 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021824799999999999 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98062 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4407296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.179252314000003, 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13404179099999999, 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98062 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445789+0000", 2024-09-18T00:59:37.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037705569999999999 2024-09-18T00:59:37.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445805+0000", 2024-09-18T00:59:37.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.558e-05 2024-09-18T00:59:37.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445846+0000", 2024-09-18T00:59:37.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:59:37.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445871+0000", 2024-09-18T00:59:37.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5072999999999999e-05 2024-09-18T00:59:37.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575651+0000", 2024-09-18T00:59:37.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12977999600000001 2024-09-18T00:59:37.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575835+0000", 2024-09-18T00:59:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000183931 2024-09-18T00:59:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576060+0000", 2024-09-18T00:59:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022558800000000001 2024-09-18T00:59:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.684 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98064 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2115584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.684 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:37.684 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.175287875000002, 2024-09-18T00:59:37.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13103894799999999, 2024-09-18T00:59:37.684 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98064 2024-09-18T00:59:37.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.685 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:37.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:37.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445985+0000", 2024-09-18T00:59:37.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-18T00:59:37.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445991+0000", 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446014+0000", 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3056000000000001e-05 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446030+0000", 2024-09-18T00:59:37.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5662999999999999e-05 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576622+0000", 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13059273900000001 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576805+0000", 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018304899999999999 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577022+0000", 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216481 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98065 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2402304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.175220758999998, 2024-09-18T00:59:37.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131457463, 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98065 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446051+0000", 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1260000000000001e-06 2024-09-18T00:59:37.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446057+0000", 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2040000000000002e-06 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446077+0000", 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446092+0000", 2024-09-18T00:59:37.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5124e-05 2024-09-18T00:59:37.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577098+0000", 2024-09-18T00:59:37.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13100595100000001 2024-09-18T00:59:37.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577284+0000", 2024-09-18T00:59:37.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000186152 2024-09-18T00:59:37.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577508+0000", 2024-09-18T00:59:37.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022314000000000001 2024-09-18T00:59:37.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.694 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.695 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98066 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2689024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.695 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:37.695 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.175158553000003, 2024-09-18T00:59:37.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13189522100000001, 2024-09-18T00:59:37.695 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98066 2024-09-18T00:59:37.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.696 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:37.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:37.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446113+0000", 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0240000000000001e-06 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446120+0000", 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446137+0000", 2024-09-18T00:59:37.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7479e-05 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446152+0000", 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4633e-05 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577587+0000", 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13143572100000001 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577786+0000", 2024-09-18T00:59:37.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019872999999999999 2024-09-18T00:59:37.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578008+0000", 2024-09-18T00:59:37.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022141399999999999 2024-09-18T00:59:37.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.700 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.701 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98067 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2975744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.701 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:37.701 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.175100890000003, 2024-09-18T00:59:37.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132319567, 2024-09-18T00:59:37.701 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.702 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98067 2024-09-18T00:59:37.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.702 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:37.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:37.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446171+0000", 2024-09-18T00:59:37.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:59:37.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446178+0000", 2024-09-18T00:59:37.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699999999999997e-06 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446196+0000", 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7989999999999999e-05 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446213+0000", 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7493999999999999e-05 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578083+0000", 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13186995000000001 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578266+0000", 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018321099999999999 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578490+0000", 2024-09-18T00:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022316600000000001 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98068 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3262464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.175039984000001, 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273661, 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98068 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446232+0000", 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.187e-06 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446238+0000", 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:37.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446257+0000", 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446272+0000", 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5532999999999998e-05 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578569+0000", 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132296631 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578749+0000", 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017978300000000001 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578967+0000", 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218736 2024-09-18T00:59:37.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.711 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.711 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98069 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3549184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.711 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:37.712 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.174980310999999, 2024-09-18T00:59:37.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13311771, 2024-09-18T00:59:37.712 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98069 2024-09-18T00:59:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446292+0000", 2024-09-18T00:59:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446298+0000", 2024-09-18T00:59:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4470000000000003e-06 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446340+0000", 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1705999999999997e-05 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446358+0000", 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7873999999999999e-05 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579072+0000", 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13271396399999999 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579213+0000", 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014133900000000001 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579408+0000", 2024-09-18T00:59:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019492700000000001 2024-09-18T00:59:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98070 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3835904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.174886868000002, 2024-09-18T00:59:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133517145, 2024-09-18T00:59:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98070 2024-09-18T00:59:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446385+0000", 2024-09-18T00:59:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-09-18T00:59:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446393+0000", 2024-09-18T00:59:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2629999999999997e-06 2024-09-18T00:59:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446413+0000", 2024-09-18T00:59:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9930000000000001e-05 2024-09-18T00:59:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446428+0000", 2024-09-18T00:59:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5336e-05 2024-09-18T00:59:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579494+0000", 2024-09-18T00:59:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13306637399999999 2024-09-18T00:59:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579682+0000", 2024-09-18T00:59:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018740900000000001 2024-09-18T00:59:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579901+0000", 2024-09-18T00:59:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021946300000000001 2024-09-18T00:59:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98071 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.174823701999998, 2024-09-18T00:59:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133931615, 2024-09-18T00:59:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98071 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449904+0000", 2024-09-18T00:59:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003456822 2024-09-18T00:59:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449945+0000", 2024-09-18T00:59:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0896000000000003e-05 2024-09-18T00:59:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449971+0000", 2024-09-18T00:59:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5996999999999999e-05 2024-09-18T00:59:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449986+0000", 2024-09-18T00:59:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.49e-05 2024-09-18T00:59:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579978+0000", 2024-09-18T00:59:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.129992723 2024-09-18T00:59:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580157+0000", 2024-09-18T00:59:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179006 2024-09-18T00:59:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580379+0000", 2024-09-18T00:59:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022127099999999999 2024-09-18T00:59:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98073 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4696064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.171214575, 2024-09-18T00:59:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13113503100000001, 2024-09-18T00:59:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98073 2024-09-18T00:59:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450057+0000", 2024-09-18T00:59:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-18T00:59:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450060+0000", 2024-09-18T00:59:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5329999999999999e-06 2024-09-18T00:59:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450072+0000", 2024-09-18T00:59:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1919999999999999e-05 2024-09-18T00:59:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450080+0000", 2024-09-18T00:59:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9500000000000001e-06 2024-09-18T00:59:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580951+0000", 2024-09-18T00:59:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.130871128 2024-09-18T00:59:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581057+0000", 2024-09-18T00:59:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010501399999999999 2024-09-18T00:59:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581191+0000", 2024-09-18T00:59:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013479900000000001 2024-09-18T00:59:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:37.734 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:37.735 INFO:teuthology.orchestra.run.smithi042.stdout:5265 trunc from 0xec3e8f9 to 0xdf63902 2024-09-18T00:59:37.736 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:37.737 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:37.737 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:37.738 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:37.738 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:37.738 INFO:teuthology.orchestra.run.smithi042.stdout:5268 punch from 0x359e5fe to 0x35a94ec, (0xaeee bytes) 2024-09-18T00:59:37.738 INFO:teuthology.orchestra.run.smithi042.stdout:5269 write 0x1bd1882 thru 0x1bd3260 (0x19df bytes) 2024-09-18T00:59:37.738 INFO:teuthology.orchestra.run.smithi042.stdout:5270 write 0x6cafe20 thru 0x6cb4932 (0x4b13 bytes) 2024-09-18T00:59:37.738 INFO:teuthology.orchestra.run.smithi042.stdout:5271 write 0xbe4282f thru 0xbe4ac0c (0x83de bytes) 2024-09-18T00:59:37.739 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:37.739 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:37.739 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:37.739 INFO:teuthology.orchestra.run.smithi042.stdout:5274 write 0xecb72c8 thru 0xecbf441 (0x817a bytes) 2024-09-18T00:59:37.740 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:37.740 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:37.741 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:37.733+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:37.742 INFO:teuthology.orchestra.run.smithi042.stdout:5275 read 0x59b395b thru 0x59c02c6 (0xc96c bytes) 2024-09-18T00:59:37.742 INFO:teuthology.orchestra.run.smithi042.stdout:5276 write 0xba103d1 thru 0xba1fa3b (0xf66b bytes) 2024-09-18T00:59:37.742 INFO:teuthology.orchestra.run.smithi042.stdout:5277 read 0x351ee76 thru 0x35281f3 (0x937e bytes) 2024-09-18T00:59:37.742 INFO:teuthology.orchestra.run.smithi042.stdout:5278 read 0xaf305ab thru 0xaf3149b (0xef1 bytes) 2024-09-18T00:59:37.742 INFO:teuthology.orchestra.run.smithi042.stdout:5279 punch from 0x916fe9e to 0x9172a9c, (0x2bfe bytes) 2024-09-18T00:59:37.742 INFO:teuthology.orchestra.run.smithi042.stdout:5280 punch from 0x2f9b2dd to 0x2fa5a09, (0xa72c bytes) 2024-09-18T00:59:37.743 INFO:teuthology.orchestra.run.smithi042.stdout:5281 write 0x866802b thru 0x8672ea7 (0xae7d bytes) 2024-09-18T00:59:37.743 INFO:teuthology.orchestra.run.smithi042.stdout:5283 read 0x653f7df thru 0x654906f (0x9891 bytes) 2024-09-18T00:59:37.743 INFO:teuthology.orchestra.run.smithi042.stdout:5284 trunc from 0xecbf442 to 0x6b2b291 2024-09-18T00:59:37.743 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:37.743 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:37.744 INFO:teuthology.orchestra.run.smithi042.stdout:5287 punch from 0x30aa36c to 0x30b9e0e, (0xfaa2 bytes) 2024-09-18T00:59:37.744 INFO:teuthology.orchestra.run.smithi042.stdout:5288 writesame 0x470b7a3 thru 0x470c5df data_size 0x9(0xe3d bytes) 2024-09-18T00:59:37.744 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:37.744 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:37.745 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:37.745 INFO:teuthology.orchestra.run.smithi042.stdout:5291 writesame 0x1077414 thru 0x107c8d0 data_size 0x1(0x54bd bytes) 2024-09-18T00:59:37.753 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:37.780 INFO:teuthology.orchestra.run.smithi042.stdout:5292 write 0xc6c1f4e thru 0xc6cd82d (0xb8e0 bytes) 2024-09-18T00:59:37.786 INFO:teuthology.orchestra.run.smithi042.stdout:5293 trunc from 0xc6cd82e to 0xc3bb2e5 2024-09-18T00:59:37.811 INFO:teuthology.orchestra.run.smithi042.stdout:5294 trunc from 0xc3bb2e5 to 0x4bfa7ec 2024-09-18T00:59:37.831 INFO:teuthology.orchestra.run.smithi042.stdout:5296 read 0x4835d8f thru 0x4845c1d (0xfe8f bytes) 2024-09-18T00:59:37.837 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:37.837+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:37.856 INFO:teuthology.orchestra.run.smithi042.stdout:5298 trunc from 0x4bfa7ec to 0xea07c69 2024-09-18T00:59:37.868 INFO:teuthology.orchestra.run.smithi042.stdout:5299 read 0x91d5006 thru 0x91e4eb4 (0xfeaf bytes) 2024-09-18T00:59:37.870 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:37.870 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:37.884 INFO:teuthology.orchestra.run.smithi042.stdout:5300 trunc from 0xea07c69 to 0x8d61433 2024-09-18T00:59:37.885 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:37.896 INFO:teuthology.orchestra.run.smithi042.stdout:5301 read 0x5797a30 thru 0x57a67d5 (0xeda6 bytes) 2024-09-18T00:59:37.896 INFO:teuthology.orchestra.run.smithi042.stdout:5302 punch from 0x5ede3a8 to 0x5eea41a, (0xc072 bytes) 2024-09-18T00:59:37.913 INFO:teuthology.orchestra.run.smithi042.stdout:5304 write 0xb00118e thru 0xb008b98 (0x7a0b bytes) 2024-09-18T00:59:37.919 INFO:teuthology.orchestra.run.smithi042.stdout:5305 writesame 0xa218ee4 thru 0xa21a7e2 data_size 0x9(0x18ff bytes) 2024-09-18T00:59:37.925 INFO:teuthology.orchestra.run.smithi042.stdout:5306 read 0x26c9484 thru 0x26cddf0 (0x496d bytes) 2024-09-18T00:59:37.925 INFO:teuthology.orchestra.run.smithi042.stdout:5307 read 0x4e18f12 thru 0x4e1f980 (0x6a6f bytes) 2024-09-18T00:59:37.932 INFO:teuthology.orchestra.run.smithi042.stdout:5308 trunc from 0xb008b99 to 0xd7836c5 2024-09-18T00:59:37.938 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:37.937+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:37.943 INFO:teuthology.orchestra.run.smithi042.stdout:5310 writesame 0x208b7ee thru 0x208e4f9 data_size 0x3e(0x2d0c bytes) 2024-09-18T00:59:37.947 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:37.947 INFO:teuthology.orchestra.run.smithi042.stdout:5312 write 0x7f8400a thru 0x7f8c857 (0x884e bytes) 2024-09-18T00:59:37.952 INFO:teuthology.orchestra.run.smithi042.stdout:5313 read 0xb5e2278 thru 0xb5ef668 (0xd3f1 bytes) 2024-09-18T00:59:37.952 INFO:teuthology.orchestra.run.smithi042.stdout:5314 write 0x2a4a6f3 thru 0x2a4e76e (0x407c bytes) 2024-09-18T00:59:37.957 INFO:teuthology.orchestra.run.smithi042.stdout:5316 writesame 0x6245c2c thru 0x625357c data_size 0x1(0xd951 bytes) 2024-09-18T00:59:37.998 INFO:teuthology.orchestra.run.smithi042.stdout:5319 read 0xbc79c16 thru 0xbc7c86f (0x2c5a bytes) 2024-09-18T00:59:37.998 INFO:teuthology.orchestra.run.smithi042.stdout:5320 writesame 0x584ae48 thru 0x585239c data_size 0x7(0x7555 bytes) 2024-09-18T00:59:38.005 INFO:teuthology.orchestra.run.smithi042.stdout:5321 punch from 0xe8ff6a to 0xe957ed, (0x5883 bytes) 2024-09-18T00:59:38.005 INFO:teuthology.orchestra.run.smithi042.stdout:5322 read 0xa4c252a thru 0xa4c7499 (0x4f70 bytes) 2024-09-18T00:59:38.005 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.005 INFO:teuthology.orchestra.run.smithi042.stdout:5325 read 0x9e3376e thru 0x9e35713 (0x1fa6 bytes) 2024-09-18T00:59:38.005 INFO:teuthology.orchestra.run.smithi042.stdout:5326 writesame 0x55898e1 thru 0x5595d73 data_size 0xd(0xc493 bytes) 2024-09-18T00:59:38.015 INFO:teuthology.orchestra.run.smithi042.stdout:5328 write 0x8e4a374 thru 0x8e57d3b (0xd9c8 bytes) 2024-09-18T00:59:38.020 INFO:teuthology.orchestra.run.smithi042.stdout:5329 write 0xd644245 thru 0xd651494 (0xd250 bytes) 2024-09-18T00:59:38.025 INFO:teuthology.orchestra.run.smithi042.stdout:5330 read 0xb56eb2c thru 0xb57be04 (0xd2d9 bytes) 2024-09-18T00:59:38.025 INFO:teuthology.orchestra.run.smithi042.stdout:5331 writesame 0x7b5ae43 thru 0x7b668af data_size 0x53(0xba6d bytes) 2024-09-18T00:59:38.031 INFO:teuthology.orchestra.run.smithi042.stdout:5332 flatten 2024-09-18T00:59:38.039 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:38.037+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:38.053 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:38.053 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:38.057 INFO:teuthology.orchestra.run.smithi042.stdout:5333 trunc from 0xd7836c5 to 0x8a2be90 2024-09-18T00:59:38.070 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:38.090 INFO:teuthology.orchestra.run.smithi042.stdout:5334 punch from 0x112083d to 0x112542f, (0x4bf2 bytes) 2024-09-18T00:59:38.090 INFO:teuthology.orchestra.run.smithi042.stdout:5335 punch from 0x5f82c6f to 0x5f9251d, (0xf8ae bytes) 2024-09-18T00:59:38.090 INFO:teuthology.orchestra.run.smithi042.stdout:5337 read 0x65caeba thru 0x65cffdc (0x5123 bytes) 2024-09-18T00:59:38.090 INFO:teuthology.orchestra.run.smithi042.stdout:5339 write 0x4cb6e4b thru 0x4cba916 (0x3acc bytes) 2024-09-18T00:59:38.095 INFO:teuthology.orchestra.run.smithi042.stdout:5340 write 0x5eeb346 thru 0x5eed60b (0x22c6 bytes) 2024-09-18T00:59:38.099 INFO:teuthology.orchestra.run.smithi042.stdout:5341 read 0x23aa93b thru 0x23af27f (0x4945 bytes) 2024-09-18T00:59:38.100 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.100 INFO:teuthology.orchestra.run.smithi042.stdout:5344 writesame 0x5d56467 thru 0x5d6086f data_size 0x7(0xa409 bytes) 2024-09-18T00:59:38.107 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.107 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.107 INFO:teuthology.orchestra.run.smithi042.stdout:5350 read 0x326807f thru 0x326ac33 (0x2bb5 bytes) 2024-09-18T00:59:38.107 INFO:teuthology.orchestra.run.smithi042.stdout:5351 read 0x1bf3889 thru 0x1bfe414 (0xab8c bytes) 2024-09-18T00:59:38.119 INFO:teuthology.orchestra.run.smithi042.stdout:5352 trunc from 0x8a2be90 to 0xce57e87 2024-09-18T00:59:38.133 INFO:teuthology.orchestra.run.smithi042.stdout:5353 write 0x8b0b0b2 thru 0x8b1608c (0xafdb bytes) 2024-09-18T00:59:38.138 INFO:teuthology.orchestra.run.smithi042.stdout:5354 read 0x46f658f thru 0x46fd8ac (0x731e bytes) 2024-09-18T00:59:38.138 INFO:teuthology.orchestra.run.smithi042.stdout:5355 write 0x6de4eef thru 0x6defe68 (0xaf7a bytes) 2024-09-18T00:59:38.139 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:38.137+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:38.153 INFO:teuthology.orchestra.run.smithi042.stdout:5356 trunc from 0xce57e87 to 0x92a566b 2024-09-18T00:59:38.193 INFO:teuthology.orchestra.run.smithi042.stdout:5357 writesame 0xd173483 thru 0xd179707 data_size 0x7(0x6285 bytes) 2024-09-18T00:59:38.193 INFO:teuthology.orchestra.run.smithi042.stdout:5358 read 0x4075155 thru 0x40782e3 (0x318f bytes) 2024-09-18T00:59:38.193 INFO:teuthology.orchestra.run.smithi042.stdout:5359 read 0xa803b5e thru 0xa813618 (0xfabb bytes) 2024-09-18T00:59:38.193 INFO:teuthology.orchestra.run.smithi042.stdout:5360 read 0x87a8233 thru 0x87b5f4d (0xdd1b bytes) 2024-09-18T00:59:38.193 INFO:teuthology.orchestra.run.smithi042.stdout:5361 read 0x59ca753 thru 0x59d58e4 (0xb192 bytes) 2024-09-18T00:59:38.196 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:38.196 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:38.207 INFO:teuthology.orchestra.run.smithi042.stdout:5362 trunc from 0xd179708 to 0x8303f6e 2024-09-18T00:59:38.210 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:38.222 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6] out_osds: [2, 3, 0, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:59:38.223 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:59:38.223 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-18T00:59:38.223 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-18T00:59:38.240 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:59:38.258 INFO:teuthology.orchestra.run.smithi042.stdout:5365 writesame 0xe7ff9b2 thru 0xe806efc data_size 0x1(0x754b bytes) 2024-09-18T00:59:38.258 INFO:teuthology.orchestra.run.smithi042.stdout:5367 write 0x21c63a5 thru 0x21d4aef (0xe74b bytes) 2024-09-18T00:59:38.258 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:38.257+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:38.264 INFO:teuthology.orchestra.run.smithi042.stdout:5369 trunc from 0xe806efd to 0xdc389bf 2024-09-18T00:59:38.285 INFO:teuthology.orchestra.run.smithi042.stdout:5370 trunc from 0xdc389bf to 0x8ca2ea6 2024-09-18T00:59:38.295 INFO:teuthology.orchestra.run.smithi042.stdout:5371 read 0x7d680a3 thru 0x7d70c6c (0x8bca bytes) 2024-09-18T00:59:38.295 INFO:teuthology.orchestra.run.smithi042.stdout:5374 read 0x619869f thru 0x619c201 (0x3b63 bytes) 2024-09-18T00:59:38.295 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.295 INFO:teuthology.orchestra.run.smithi042.stdout:5376 write 0x2ad9456 thru 0x2ae65e9 (0xd194 bytes) 2024-09-18T00:59:38.300 INFO:teuthology.orchestra.run.smithi042.stdout:5377 write 0x6cf7002 thru 0x6cff0eb (0x80ea bytes) 2024-09-18T00:59:38.304 INFO:teuthology.orchestra.run.smithi042.stdout:5378 writesame 0x1eaa8fe thru 0x1eb600f data_size 0x6(0xb712 bytes) 2024-09-18T00:59:38.323 INFO:teuthology.orchestra.run.smithi042.stdout:5379 trunc from 0x8ca2ea6 to 0x43cf46f 2024-09-18T00:59:38.342 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:38.341+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:38.351 INFO:teuthology.orchestra.run.smithi042.stdout:5380 read 0x1dd93d0 thru 0x1ddaadd (0x170e bytes) 2024-09-18T00:59:38.351 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.351 INFO:teuthology.orchestra.run.smithi042.stdout:5382 read 0x43a01ad thru 0x43a068b (0x4df bytes) 2024-09-18T00:59:38.351 INFO:teuthology.orchestra.run.smithi042.stdout:5383 read 0x201bd6a thru 0x2027887 (0xbb1e bytes) 2024-09-18T00:59:38.351 INFO:teuthology.orchestra.run.smithi042.stdout:5384 read 0x3b75e29 thru 0x3b7ac62 (0x4e3a bytes) 2024-09-18T00:59:38.361 INFO:teuthology.orchestra.run.smithi042.stdout:5385 write 0x5e6f962 thru 0x5e74bab (0x524a bytes) 2024-09-18T00:59:38.367 INFO:teuthology.orchestra.run.smithi042.stdout:5386 write 0x47abba0 thru 0x47b93ae (0xd80f bytes) 2024-09-18T00:59:38.371 INFO:teuthology.orchestra.run.smithi042.stdout:5387 write 0x2dc7ac6 thru 0x2dcc948 (0x4e83 bytes) 2024-09-18T00:59:38.374 INFO:teuthology.orchestra.run.smithi042.stdout:5389 punch from 0xaba0d2 to 0xabd459, (0x3387 bytes) 2024-09-18T00:59:38.374 INFO:teuthology.orchestra.run.smithi042.stdout:5391 punch from 0x12cfd29 to 0x12dce0f, (0xd0e6 bytes) 2024-09-18T00:59:38.374 INFO:teuthology.orchestra.run.smithi042.stdout:5393 punch from 0x3baf04c to 0x3bbc32f, (0xd2e3 bytes) 2024-09-18T00:59:38.374 INFO:teuthology.orchestra.run.smithi042.stdout:5394 punch from 0x1712f16 to 0x171e4b7, (0xb5a1 bytes) 2024-09-18T00:59:38.374 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.375 INFO:teuthology.orchestra.run.smithi042.stdout:5397 writesame 0x47390f4 thru 0x4743c18 data_size 0xb(0xab25 bytes) 2024-09-18T00:59:38.382 INFO:teuthology.orchestra.run.smithi042.stdout:5398 read 0x90eb1e thru 0x910354 (0x1837 bytes) 2024-09-18T00:59:38.382 INFO:teuthology.orchestra.run.smithi042.stdout:5400 read 0xa1292d thru 0xa16134 (0x3808 bytes) 2024-09-18T00:59:38.383 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:38.383 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:38.391 INFO:teuthology.orchestra.run.smithi042.stdout:5401 writesame 0x7172d16 thru 0x7177be1 data_size 0x29(0x4ecc bytes) 2024-09-18T00:59:38.391 INFO:teuthology.orchestra.run.smithi042.stdout:5402 writesame 0x2ceafcb thru 0x2cf9889 data_size 0x1(0xe8bf bytes) 2024-09-18T00:59:38.401 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:38.419 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-18T00:59:38.428 INFO:teuthology.orchestra.run.smithi042.stdout:5404 write 0x9a17f6 thru 0x9a63fe (0x4c09 bytes) 2024-09-18T00:59:38.438 INFO:teuthology.orchestra.run.smithi042.stdout:5405 trunc from 0x7177be2 to 0x273768b 2024-09-18T00:59:38.443 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:38.441+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:38.459 INFO:teuthology.orchestra.run.smithi042.stdout:5407 write 0x3f75be8 thru 0x3f84de2 (0xf1fb bytes) 2024-09-18T00:59:38.463 INFO:teuthology.orchestra.run.smithi042.stdout:5408 write 0x277bde5 thru 0x278915a (0xd376 bytes) 2024-09-18T00:59:38.468 INFO:teuthology.orchestra.run.smithi042.stdout:5409 write 0x2f0aff1 thru 0x2f17460 (0xc470 bytes) 2024-09-18T00:59:38.488 INFO:teuthology.orchestra.run.smithi042.stdout:5410 write 0xa19751d thru 0xa19d0d0 (0x5bb4 bytes) 2024-09-18T00:59:38.492 INFO:teuthology.orchestra.run.smithi042.stdout:5411 read 0x120b829 thru 0x1217cc7 (0xc49f bytes) 2024-09-18T00:59:38.492 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.492 INFO:teuthology.orchestra.run.smithi042.stdout:5414 write 0x9f230da thru 0x9f2f1ec (0xc113 bytes) 2024-09-18T00:59:38.496 INFO:teuthology.orchestra.run.smithi042.stdout:5416 read 0x1e1a9dd thru 0x1e260eb (0xb70f bytes) 2024-09-18T00:59:38.496 INFO:teuthology.orchestra.run.smithi042.stdout:5417 write 0x4d2ac18 thru 0x4d32373 (0x775c bytes) 2024-09-18T00:59:38.500 INFO:teuthology.orchestra.run.smithi042.stdout:5418 read 0x2f60618 thru 0x2f6ac57 (0xa640 bytes) 2024-09-18T00:59:38.500 INFO:teuthology.orchestra.run.smithi042.stdout:5419 read 0x2ac9467 thru 0x2acd6f5 (0x428f bytes) 2024-09-18T00:59:38.500 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.500 INFO:teuthology.orchestra.run.smithi042.stdout:5421 write 0x7842f7 thru 0x7917c6 (0xd4d0 bytes) 2024-09-18T00:59:38.503 INFO:teuthology.orchestra.run.smithi042.stdout:5422 read 0x6275de5 thru 0x6276c2b (0xe47 bytes) 2024-09-18T00:59:38.517 INFO:teuthology.orchestra.run.smithi042.stdout:5423 write 0xea5b808 thru 0xea69441 (0xdc3a bytes) 2024-09-18T00:59:38.521 INFO:teuthology.orchestra.run.smithi042.stdout:5424 writesame 0xa3e8a9f thru 0xa3f6eaa data_size 0x54(0xe40c bytes) 2024-09-18T00:59:38.527 INFO:teuthology.orchestra.run.smithi042.stdout:5426 write 0xe8e8f40 thru 0xe8ecc25 (0x3ce6 bytes) 2024-09-18T00:59:38.532 INFO:teuthology.orchestra.run.smithi042.stdout:5427 read 0x69384c6 thru 0x694740f (0xef4a bytes) 2024-09-18T00:59:38.532 INFO:teuthology.orchestra.run.smithi042.stdout:5428 write 0x296c94c thru 0x296d22d (0x8e2 bytes) 2024-09-18T00:59:38.534 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:38.535 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:38.537 INFO:teuthology.orchestra.run.smithi042.stdout:5429 read 0xdfdfbd2 thru 0xdfeb0b2 (0xb4e1 bytes) 2024-09-18T00:59:38.544 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:38.541+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:38.544 INFO:teuthology.orchestra.run.smithi042.stdout:5431 trunc from 0xea69442 to 0x990c3a4 2024-09-18T00:59:38.550 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:38.569 INFO:teuthology.orchestra.run.smithi042.stdout:5433 write 0xb81ade3 thru 0xb81e481 (0x369f bytes) 2024-09-18T00:59:38.573 INFO:teuthology.orchestra.run.smithi042.stdout:5434 writesame 0x7d2baeb thru 0x7d38b75 data_size 0x1(0xd08b bytes) 2024-09-18T00:59:38.606 INFO:teuthology.orchestra.run.smithi042.stdout:5436 write 0x387c05f thru 0x38826ac (0x664e bytes) 2024-09-18T00:59:38.612 INFO:teuthology.orchestra.run.smithi042.stdout:5437 punch from 0x965bd62 to 0x9667d1e, (0xbfbc bytes) 2024-09-18T00:59:38.612 INFO:teuthology.orchestra.run.smithi042.stdout:5438 write 0x580cb76 thru 0x581803f (0xb4ca bytes) 2024-09-18T00:59:38.614 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-18T00:59:38.615 INFO:teuthology.orchestra.run.smithi042.stdout:5440 read 0x909ad0b thru 0x909cbeb (0x1ee1 bytes) 2024-09-18T00:59:38.615 INFO:teuthology.orchestra.run.smithi042.stdout:5443 write 0x2e3289e thru 0x2e3d112 (0xa875 bytes) 2024-09-18T00:59:38.618 INFO:teuthology.orchestra.run.smithi042.stdout:5444 punch from 0x35dc603 to 0x35e3c07, (0x7604 bytes) 2024-09-18T00:59:38.619 INFO:teuthology.orchestra.run.smithi042.stdout:5446 write 0x4bb67e4 thru 0x4bb768d (0xeaa bytes) 2024-09-18T00:59:38.622 INFO:teuthology.orchestra.run.smithi042.stdout:5447 read 0xae1c5fc thru 0xae1d554 (0xf59 bytes) 2024-09-18T00:59:38.622 INFO:teuthology.orchestra.run.smithi042.stdout:5449 read 0x6615450 thru 0x6620022 (0xabd3 bytes) 2024-09-18T00:59:38.629 INFO:teuthology.orchestra.run.smithi042.stdout:5450 write 0xc35274c thru 0xc355c98 (0x354d bytes) 2024-09-18T00:59:38.642 INFO:teuthology.orchestra.run.smithi042.stdout:5452 trunc from 0xc355c99 to 0x58a9475 2024-09-18T00:59:38.644 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:38.641+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:38.662 INFO:teuthology.orchestra.run.smithi042.stdout:5453 trunc from 0x58a9475 to 0x36e5942 2024-09-18T00:59:38.687 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:38.687 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:38.687 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:5456 write 0x7895fa8 thru 0x789ccdb (0x6d34 bytes) 2024-09-18T00:59:38.702 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:38.707 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.707 INFO:teuthology.orchestra.run.smithi042.stdout:5458 write 0x339059d thru 0x339644d (0x5eb1 bytes) 2024-09-18T00:59:38.713 INFO:teuthology.orchestra.run.smithi042.stdout:5459 punch from 0x3ce5278 to 0x3cec7c1, (0x7549 bytes) 2024-09-18T00:59:38.713 INFO:teuthology.orchestra.run.smithi042.stdout:5461 read 0x2239fe8 thru 0x223a5db (0x5f4 bytes) 2024-09-18T00:59:38.713 INFO:teuthology.orchestra.run.smithi042.stdout:5464 read 0x6dc228d thru 0x6dced67 (0xcadb bytes) 2024-09-18T00:59:38.713 INFO:teuthology.orchestra.run.smithi042.stdout:5465 punch from 0x43c1a93 to 0x43cf69e, (0xdc0b bytes) 2024-09-18T00:59:38.713 INFO:teuthology.orchestra.run.smithi042.stdout:5466 trunc from 0x789ccdc to 0x7e3077a 2024-09-18T00:59:38.732 INFO:teuthology.orchestra.run.smithi042.stdout:5467 trunc from 0x7e3077a to 0x15990f3 2024-09-18T00:59:38.748 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:38.749+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:38.751 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.751 INFO:teuthology.orchestra.run.smithi042.stdout:5471 read 0x128fb8e thru 0x129697d (0x6df0 bytes) 2024-09-18T00:59:38.751 INFO:teuthology.orchestra.run.smithi042.stdout:5472 read 0x582200 thru 0x583585 (0x1386 bytes) 2024-09-18T00:59:38.751 INFO:teuthology.orchestra.run.smithi042.stdout:5473 punch from 0x93e840 to 0x941acf, (0x328f bytes) 2024-09-18T00:59:38.751 INFO:teuthology.orchestra.run.smithi042.stdout:5474 punch from 0x21abee to 0x21e8bc, (0x3cce bytes) 2024-09-18T00:59:38.756 INFO:teuthology.orchestra.run.smithi042.stdout:5475 trunc from 0x15990f3 to 0x504c6a7 2024-09-18T00:59:38.784 INFO:teuthology.orchestra.run.smithi042.stdout:5476 write 0xe8a7a2b thru 0xe8b6dfc (0xf3d2 bytes) 2024-09-18T00:59:38.788 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.788 INFO:teuthology.orchestra.run.smithi042.stdout:5478 read 0x8dcf50f thru 0x8dd5f5b (0x6a4d bytes) 2024-09-18T00:59:38.788 INFO:teuthology.orchestra.run.smithi042.stdout:5479 read 0x865eb15 thru 0x865f25c (0x748 bytes) 2024-09-18T00:59:38.789 INFO:teuthology.orchestra.run.smithi042.stdout:5480 read 0x25099d3 thru 0x2516db9 (0xd3e7 bytes) 2024-09-18T00:59:38.789 INFO:teuthology.orchestra.run.smithi042.stdout:5481 punch from 0x250a98d to 0x25157ec, (0xae5f bytes) 2024-09-18T00:59:38.789 INFO:teuthology.orchestra.run.smithi042.stdout:5482 write 0x22ce3a5 thru 0x22d4f89 (0x6be5 bytes) 2024-09-18T00:59:38.795 INFO:teuthology.orchestra.run.smithi042.stdout:5483 trunc from 0xe8b6dfd to 0xc7a7c14 2024-09-18T00:59:38.798 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-18T00:59:38.808 INFO:teuthology.orchestra.run.smithi042.stdout:5484 write 0xa53b863 thru 0xa544920 (0x90be bytes) 2024-09-18T00:59:38.815 INFO:teuthology.orchestra.run.smithi042.stdout:5485 read 0x63c20d6 thru 0x63cb134 (0x905f bytes) 2024-09-18T00:59:38.816 INFO:teuthology.orchestra.run.smithi042.stdout:5487 write 0x7dfa5a6 thru 0x7e03e91 (0x98ec bytes) 2024-09-18T00:59:38.823 INFO:teuthology.orchestra.run.smithi042.stdout:5489 punch from 0x4d2a0e7 to 0x4d34767, (0xa680 bytes) 2024-09-18T00:59:38.823 INFO:teuthology.orchestra.run.smithi042.stdout:5491 write 0x59ba7af thru 0x59c8a2e (0xe280 bytes) 2024-09-18T00:59:38.831 INFO:teuthology.orchestra.run.smithi042.stdout:5493 writesame 0x71de952 thru 0x71edc33 data_size 0x81(0xf2e2 bytes) 2024-09-18T00:59:38.841 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:38.841 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:38.845 INFO:teuthology.orchestra.run.smithi042.stdout:5494 write 0xd3d5c3a thru 0xd3e38ac (0xdc73 bytes) 2024-09-18T00:59:38.846 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:38.845+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:38.851 INFO:teuthology.orchestra.run.smithi042.stdout:5495 read 0x51866c2 thru 0x5189d98 (0x36d7 bytes) 2024-09-18T00:59:38.851 INFO:teuthology.orchestra.run.smithi042.stdout:5497 read 0x9d24e49 thru 0x9d28ebe (0x4076 bytes) 2024-09-18T00:59:38.857 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:38.860 INFO:teuthology.orchestra.run.smithi042.stdout:5498 write 0xdafbadc thru 0xdb089c4 (0xcee9 bytes) 2024-09-18T00:59:38.867 INFO:teuthology.orchestra.run.smithi042.stdout:5499 write 0x5a38976 thru 0x5a3fc8f (0x731a bytes) 2024-09-18T00:59:38.870 INFO:teuthology.orchestra.run.smithi042.stdout:5500 writesame 0x3d16c91 thru 0x3d21a55 data_size 0x29(0xadc5 bytes) 2024-09-18T00:59:38.875 INFO:teuthology.orchestra.run.smithi042.stdout:5503 read 0x50c1847 thru 0x50cf5cd (0xdd87 bytes) 2024-09-18T00:59:38.875 INFO:teuthology.orchestra.run.smithi042.stdout:5504 punch from 0x28c87f8 to 0x28cf6d9, (0x6ee1 bytes) 2024-09-18T00:59:38.875 INFO:teuthology.orchestra.run.smithi042.stdout:5506 punch from 0x734b1fc to 0x734ebf4, (0x39f8 bytes) 2024-09-18T00:59:38.875 INFO:teuthology.orchestra.run.smithi042.stdout:5507 write 0x2670c49 thru 0x2672b96 (0x1f4e bytes) 2024-09-18T00:59:38.879 INFO:teuthology.orchestra.run.smithi042.stdout:5508 writesame 0x3046340 thru 0x3048164 data_size 0x1(0x1e25 bytes) 2024-09-18T00:59:38.888 INFO:teuthology.orchestra.run.smithi042.stdout:5509 trunc from 0xdb089c5 to 0xd9fedb4 2024-09-18T00:59:38.904 INFO:teuthology.orchestra.run.smithi042.stdout:5510 write 0xaaa70c1 thru 0xaaae557 (0x7497 bytes) 2024-09-18T00:59:38.925 INFO:teuthology.orchestra.run.smithi042.stdout:5511 trunc from 0xd9fedb4 to 0x27bffa7 2024-09-18T00:59:38.944 INFO:teuthology.orchestra.run.smithi042.stdout:5516 read 0x14198a5 thru 0x14259bc (0xc118 bytes) 2024-09-18T00:59:38.944 INFO:teuthology.orchestra.run.smithi042.stdout:5517 read 0x223f267 thru 0x22446a6 (0x5440 bytes) 2024-09-18T00:59:38.945 INFO:teuthology.orchestra.run.smithi042.stdout:5518 read 0x7bcd42 thru 0x7ca032 (0xd2f1 bytes) 2024-09-18T00:59:38.945 INFO:teuthology.orchestra.run.smithi042.stdout:5519 punch from 0x1cc6c97 to 0x1cce7dc, (0x7b45 bytes) 2024-09-18T00:59:38.945 INFO:teuthology.orchestra.run.smithi042.stdout:5520 punch from 0x1233de6 to 0x123b9e7, (0x7c01 bytes) 2024-09-18T00:59:38.945 INFO:teuthology.orchestra.run.smithi042.stdout:5522 read 0x1ecc48b thru 0x1ed3099 (0x6c0f bytes) 2024-09-18T00:59:38.946 INFO:teuthology.orchestra.run.smithi042.stdout:5523 punch from 0x138f23b to 0x139066f, (0x1434 bytes) 2024-09-18T00:59:38.946 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.946 INFO:teuthology.orchestra.run.smithi042.stdout:5525 punch from 0x1445258 to 0x144e249, (0x8ff1 bytes) 2024-09-18T00:59:38.946 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:38.947 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:38.945+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:38.970 INFO:teuthology.orchestra.run.smithi042.stdout:5529 writesame 0xc50a8aa thru 0xc50ae6f data_size 0x1(0x5c6 bytes) 2024-09-18T00:59:38.970 INFO:teuthology.orchestra.run.smithi042.stdout:5531 read 0x968d640 thru 0x96957aa (0x816b bytes) 2024-09-18T00:59:38.970 INFO:teuthology.orchestra.run.smithi042.stdout:5532 write 0x1d8fe58 thru 0x1d9355c (0x3705 bytes) 2024-09-18T00:59:38.995 INFO:teuthology.orchestra.run.smithi042.stdout:5533 trunc from 0xc50ae70 to 0xa42782 2024-09-18T00:59:39.012 INFO:teuthology.orchestra.run.smithi042.stdout:5535 punch from 0x9f9584 to 0xa02938, (0x93b4 bytes) 2024-09-18T00:59:39.028 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:39.028 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:39.042 INFO:teuthology.orchestra.run.smithi042.stdout:5536 write 0xea77fa8 thru 0xea7f7a1 (0x77fa bytes) 2024-09-18T00:59:39.043 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:39.047 INFO:teuthology.orchestra.run.smithi042.stdout:5538 writesame 0x96a1dca thru 0x96abd3b data_size 0x3(0x9f72 bytes) 2024-09-18T00:59:39.058 INFO:teuthology.orchestra.run.smithi042.stdout:5540 punch from 0x95a4e93 to 0x95a779f, (0x290c bytes) 2024-09-18T00:59:39.061 INFO:teuthology.orchestra.run.smithi042.stdout:5541 trunc from 0xea7f7a2 to 0xbfc7f70 2024-09-18T00:59:39.080 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:39.080 INFO:teuthology.orchestra.run.smithi042.stdout:5544 write 0x4f0604c thru 0x4f0e159 (0x810e bytes) 2024-09-18T00:59:39.085 INFO:teuthology.orchestra.run.smithi042.stdout:5545 punch from 0x2af8342 to 0x2afcc19, (0x48d7 bytes) 2024-09-18T00:59:39.085 INFO:teuthology.orchestra.run.smithi042.stdout:5548 write 0xa042d00 thru 0xa049add (0x6dde bytes) 2024-09-18T00:59:39.088 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:39.089+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:39.090 INFO:teuthology.orchestra.run.smithi042.stdout:5549 read 0xb94241c thru 0xb94a13f (0x7d24 bytes) 2024-09-18T00:59:39.090 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:39.103 INFO:teuthology.orchestra.run.smithi042.stdout:5551 trunc from 0xbfc7f70 to 0x5f354de 2024-09-18T00:59:39.120 INFO:teuthology.orchestra.run.smithi042.stdout:5552 read 0xe7cf7 thru 0xed4e7 (0x57f1 bytes) 2024-09-18T00:59:39.121 INFO:teuthology.orchestra.run.smithi042.stdout:5554 punch from 0x13d92ad to 0x13dd616, (0x4369 bytes) 2024-09-18T00:59:39.121 INFO:teuthology.orchestra.run.smithi042.stdout:5555 read 0x319f0d0 thru 0x31a5ad0 (0x6a01 bytes) 2024-09-18T00:59:39.125 INFO:teuthology.orchestra.run.smithi042.stdout:5556 trunc from 0x5f354de to 0x7af0b8a 2024-09-18T00:59:39.138 INFO:teuthology.orchestra.run.smithi042.stdout:5560 read 0x17f9c58 thru 0x1807a86 (0xde2f bytes) 2024-09-18T00:59:39.138 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:39.138 INFO:teuthology.orchestra.run.smithi042.stdout:5562 write 0x4f85bbe thru 0x4f8e53f (0x8982 bytes) 2024-09-18T00:59:39.147 INFO:teuthology.orchestra.run.smithi042.stdout:5563 read 0x79c5a6f thru 0x79c8edd (0x346f bytes) 2024-09-18T00:59:39.148 INFO:teuthology.orchestra.run.smithi042.stdout:5564 punch from 0x658b6c8 to 0x6598df7, (0xd72f bytes) 2024-09-18T00:59:39.148 INFO:teuthology.orchestra.run.smithi042.stdout:5566 write 0x1f233b8 thru 0x1f25906 (0x254f bytes) 2024-09-18T00:59:39.148 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:39.145+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:39.172 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:39.172 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:39.175 INFO:teuthology.orchestra.run.smithi042.stdout:5568 write 0xa7c8714 thru 0xa7c8b61 (0x44e bytes) 2024-09-18T00:59:39.182 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:39.182 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:39.182 INFO:teuthology.orchestra.run.smithi042.stdout:5574 read 0x4630dbc thru 0x46323ee (0x1633 bytes) 2024-09-18T00:59:39.182 INFO:teuthology.orchestra.run.smithi042.stdout:5575 writesame 0xef7681 thru 0xefc800 data_size 0x40(0x5180 bytes) 2024-09-18T00:59:39.188 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:39.190 INFO:teuthology.orchestra.run.smithi042.stdout:5576 read 0x25d2b51 thru 0x25d8c5d (0x610d bytes) 2024-09-18T00:59:39.190 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:39.190 INFO:teuthology.orchestra.run.smithi042.stdout:5578 punch from 0xdf63d6 to 0xdfc51a, (0x6144 bytes) 2024-09-18T00:59:39.190 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:39.194 INFO:teuthology.orchestra.run.smithi042.stdout:5580 trunc from 0xa7c8b62 to 0x912d7bb 2024-09-18T00:59:39.217 INFO:teuthology.orchestra.run.smithi042.stdout:5582 read 0x469b489 thru 0x46a9bfa (0xe772 bytes) 2024-09-18T00:59:39.222 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:59:39.236 INFO:teuthology.orchestra.run.smithi042.stdout:5583 write 0xd154df0 thru 0xd1645b4 (0xf7c5 bytes) 2024-09-18T00:59:39.245 INFO:teuthology.orchestra.run.smithi042.stdout:5586 write 0x54b9a30 thru 0x54bf6d9 (0x5caa bytes) 2024-09-18T00:59:39.249 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:39.261 INFO:teuthology.orchestra.run.smithi042.stdout:5589 trunc from 0xd1645b5 to 0x5330fb2 2024-09-18T00:59:39.274 INFO:teuthology.orchestra.run.smithi042.stdout:5590 read 0x3ec4d13 thru 0x3ecac08 (0x5ef6 bytes) 2024-09-18T00:59:39.277 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:39.277+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:39.287 INFO:teuthology.orchestra.run.smithi042.stdout:5593 write 0x93e6732 thru 0x93f217d (0xba4c bytes) 2024-09-18T00:59:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:5594 read 0x38a2f7a thru 0x38b042a (0xd4b1 bytes) 2024-09-18T00:59:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:5595 punch from 0x45e18a4 to 0x45e2f44, (0x16a0 bytes) 2024-09-18T00:59:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:5596 read 0x4f613d3 thru 0x4f6242c (0x105a bytes) 2024-09-18T00:59:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:5597 write 0x5edd617 thru 0x5ee8cdd (0xb6c7 bytes) 2024-09-18T00:59:39.313 INFO:teuthology.orchestra.run.smithi042.stdout:5599 writesame 0xe02afe6 thru 0xe02bb06 data_size 0xb(0xb21 bytes) 2024-09-18T00:59:39.313 INFO:teuthology.orchestra.run.smithi042.stdout:5600 read 0x6344923 thru 0x634adf1 (0x64cf bytes) 2024-09-18T00:59:39.313 INFO:teuthology.orchestra.run.smithi042.stdout:5601 writesame 0x58e22a4 thru 0x58e3b6d data_size 0x2(0x18ca bytes) 2024-09-18T00:59:39.319 INFO:teuthology.orchestra.run.smithi042.stdout:5602 read 0x373d6b8 thru 0x374780e (0xa157 bytes) 2024-09-18T00:59:39.319 INFO:teuthology.orchestra.run.smithi042.stdout:5604 punch from 0x497dcac to 0x4989273, (0xb5c7 bytes) 2024-09-18T00:59:39.319 INFO:teuthology.orchestra.run.smithi042.stdout:5605 writesame 0x764e5a3 thru 0x7656417 data_size 0x63(0x7e75 bytes) 2024-09-18T00:59:39.324 INFO:teuthology.orchestra.run.smithi042.stdout:5607 write 0x32e9b4b thru 0x32ee92c (0x4de2 bytes) 2024-09-18T00:59:39.329 INFO:teuthology.orchestra.run.smithi042.stdout:5610 read 0x99f6d44 thru 0x99f7d00 (0xfbd bytes) 2024-09-18T00:59:39.329 INFO:teuthology.orchestra.run.smithi042.stdout:5612 writesame 0x795174b thru 0x795681d data_size 0x39(0x50d3 bytes) 2024-09-18T00:59:39.334 INFO:teuthology.orchestra.run.smithi042.stdout:5614 punch from 0x3dbdd75 to 0x3dc3940, (0x5bcb bytes) 2024-09-18T00:59:39.349 INFO:teuthology.orchestra.run.smithi042.stdout:5615 trunc from 0xe02bb07 to 0x5fb6bce 2024-09-18T00:59:39.350 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:39.349+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:39.370 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:39.371 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:39.385 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:39.399 INFO:teuthology.orchestra.run.smithi042.stdout:5616 write 0xca77b4c thru 0xca80b73 (0x9028 bytes) 2024-09-18T00:59:39.408 INFO:teuthology.orchestra.run.smithi042.stdout:5617 read 0xbbe7c46 thru 0xbbf12c6 (0x9681 bytes) 2024-09-18T00:59:39.408 INFO:teuthology.orchestra.run.smithi042.stdout:5618 read 0x20ff4f3 thru 0x21009ad (0x14bb bytes) 2024-09-18T00:59:39.408 INFO:teuthology.orchestra.run.smithi042.stdout:5623 write 0xea975d thru 0xea9bc7 (0x46b bytes) 2024-09-18T00:59:39.413 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:39.413 INFO:teuthology.orchestra.run.smithi042.stdout:5626 read 0x2d823ef thru 0x2d8754d (0x515f bytes) 2024-09-18T00:59:39.413 INFO:teuthology.orchestra.run.smithi042.stdout:5627 read 0x7974796 thru 0x797c75e (0x7fc9 bytes) 2024-09-18T00:59:39.413 INFO:teuthology.orchestra.run.smithi042.stdout:5628 read 0xad84d7b thru 0xad93462 (0xe6e8 bytes) 2024-09-18T00:59:39.413 INFO:teuthology.orchestra.run.smithi042.stdout:5629 punch from 0xc1e4ea1 to 0xc1e5003, (0x162 bytes) 2024-09-18T00:59:39.421 INFO:teuthology.orchestra.run.smithi042.stdout:5630 trunc from 0xca80b74 to 0x950fb46 2024-09-18T00:59:39.441 INFO:teuthology.orchestra.run.smithi042.stdout:5631 read 0x539b9d9 thru 0x53a51e5 (0x980d bytes) 2024-09-18T00:59:39.441 INFO:teuthology.orchestra.run.smithi042.stdout:5632 clone 42 order 19 su 16384 sc 8 2024-09-18T00:59:39.473 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:39.473+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:39.551 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:39.551 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:39.551 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:39.549+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:39.565 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:39.652 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:39.653+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:39.734 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:39.735 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:39.749 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:39.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:39.794 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:39.793+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:39.854 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:39.853+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:39.887 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:39.887 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:39.902 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:39.949 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:39.967 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:39.967 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:39.967 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:39.965+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:39.983 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:39.984 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:40.000 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:40.000 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:40.022 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:40.042 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:40.043 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:40.056 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:40.053+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:40.064 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:40.064 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:40.065 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:40.066 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:40.080 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:40.087 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:40.088 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:40.158 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:40.157+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:40.211 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:40.211 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:40.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:40.211 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:40.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.211 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.211 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:40.211 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 111.143290936, 2024-09-18T00:59:40.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T00:59:40.211 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.211 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T00:59:40.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T00:59:40.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T00:59:40.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T00:59:40.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.250757116999999, 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.217 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.221 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.501690326999999, 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T00:59:40.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T00:59:40.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T00:59:40.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T00:59:40.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.491315871000001, 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.491243188999999, 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.231 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.231 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.231 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T00:59:40.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.231 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:40.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.491212322999999, 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T00:59:40.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.491159548999999, 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T00:59:40.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T00:59:40.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T00:59:40.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T00:59:40.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T00:59:40.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.491133383000001, 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.491108228000002, 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T00:59:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T00:59:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T00:59:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T00:59:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T00:59:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.490684916999999, 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T00:59:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.490646539, 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T00:59:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.490621774000001, 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.483827009999999, 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T00:59:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T00:59:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T00:59:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T00:59:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.483801410000002, 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T00:59:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T00:59:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.483778066999999, 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T00:59:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T00:59:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T00:59:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T00:59:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T00:59:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T00:59:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T00:59:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T00:59:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.483755628000001, 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T00:59:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.483734855000002, 2024-09-18T00:59:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T00:59:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.483713828999999, 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T00:59:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.483673246999999, 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:59:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T00:59:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.483639148000002, 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.306 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:40.308 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:40.309 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:40.305+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:40.409 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:40.409+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:40.439 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:40.439 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:40.440 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:40.440 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:40.456 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:40.509 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:40.509+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:40.528 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-18T00:59:40.599 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:40.599 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:40.600 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:40.600 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:40.600 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:40.610 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:40.609+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:40.618 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:40.711 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:40.709+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:40.812 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:40.809+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:40.832 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 137.54109543000001, 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:59:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:59:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:59:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:59:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7437 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 917504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.344754377, 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045542844999999998, 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7437 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485661+0000", 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6699999999999999e-07 2024-09-18T00:59:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485664+0000", 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485670+0000", 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3759999999999996e-06 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485675+0000", 2024-09-18T00:59:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4079999999999997e-06 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531083+0000", 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045407760999999998 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531147+0000", 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4165999999999995e-05 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531204+0000", 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.66e-05 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.34473294, 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:59:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.344637613, 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:59:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.34461217400001, 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:59:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:59:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.34458965100001, 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:59:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.344566211, 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:59:40.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.344542401, 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:59:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.34452003600001, 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.344493419, 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:59:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.340320905, 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:59:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.34027534000001, 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:59:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:59:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.340248999, 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:59:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.340226329, 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:59:40.897 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:59:40.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.340207956, 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.340188168, 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:59:40.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:59:40.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:40.911 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.340161298, 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:40.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:59:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.263970829000002, 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.916 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T00:59:40.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T00:59:40.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.852905733, 2024-09-18T00:59:40.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:40.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T00:59:40.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.928 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:140635 3.d 3:b08054e4:::rbd_object_map.117ced55205b.0000000000000023:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4917923389999999, 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86222047999999996, 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:40.929 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 140635 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338625+0000", 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338634+0000", 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9099999999999994e-06 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338657+0000", 2024-09-18T00:59:40.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3672999999999998e-05 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338675+0000", 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7589999999999999e-05 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338767+0000", 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1896000000000002e-05 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338776+0000", 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.38e-06 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200213+0000", 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86143668200000001 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200432+0000", 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021884199999999999 2024-09-18T00:59:40.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:40.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:40.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:40.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200844+0000", 2024-09-18T00:59:40.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000411887 2024-09-18T00:59:40.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.934 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:40.934 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:40.934 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:40.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:40.938 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:40.937+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:41.036 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:41.037+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:41.043 INFO:teuthology.orchestra.run.smithi042.stdout:5633 read 0x112456e thru 0x11337a4 (0xf237 bytes) 2024-09-18T00:59:41.050 INFO:teuthology.orchestra.run.smithi042.stdout:5634 punch from 0x8fdaa52 to 0x8fdb43e, (0x9ec bytes) 2024-09-18T00:59:41.050 INFO:teuthology.orchestra.run.smithi042.stdout:5635 write 0x928e89c thru 0x929d1d7 (0xe93c bytes) 2024-09-18T00:59:41.068 INFO:teuthology.orchestra.run.smithi042.stdout:5636 punch from 0x27a39ff to 0x27aa516, (0x6b17 bytes) 2024-09-18T00:59:41.068 INFO:teuthology.orchestra.run.smithi042.stdout:5637 write 0x11c9654 thru 0x11cedb2 (0x575f bytes) 2024-09-18T00:59:41.076 INFO:teuthology.orchestra.run.smithi042.stdout:5638 write 0x1c506a thru 0x1c6a2a (0x19c1 bytes) 2024-09-18T00:59:41.091 INFO:teuthology.orchestra.run.smithi042.stdout:5640 trunc from 0x950fb46 to 0x42d37f9 2024-09-18T00:59:41.103 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:41.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:41.103 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:41.103 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:41.119 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:41.129 INFO:teuthology.orchestra.run.smithi042.stdout:5642 punch from 0x1ea247 to 0x1f90ff, (0xeeb8 bytes) 2024-09-18T00:59:41.129 INFO:teuthology.orchestra.run.smithi042.stdout:5643 punch from 0xde3cf1 to 0xde78ed, (0x3bfc bytes) 2024-09-18T00:59:41.133 INFO:teuthology.orchestra.run.smithi042.stdout:5645 trunc from 0x42d37f9 to 0x75d251b 2024-09-18T00:59:41.137 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:41.133+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:41.144 INFO:teuthology.orchestra.run.smithi042.stdout:5646 write 0x27fcf5a thru 0x2801241 (0x42e8 bytes) 2024-09-18T00:59:41.151 INFO:teuthology.orchestra.run.smithi042.stdout:5647 read 0x6d60f3a thru 0x6d6e658 (0xd71f bytes) 2024-09-18T00:59:41.151 INFO:teuthology.orchestra.run.smithi042.stdout:5648 punch from 0x4820965 to 0x48302ee, (0xf989 bytes) 2024-09-18T00:59:41.151 INFO:teuthology.orchestra.run.smithi042.stdout:5650 read 0x40d2805 thru 0x40dd0fa (0xa8f6 bytes) 2024-09-18T00:59:41.152 INFO:teuthology.orchestra.run.smithi042.stdout:5652 writesame 0x30e6be1 thru 0x30ec748 data_size 0xf(0x5b68 bytes) 2024-09-18T00:59:41.165 INFO:teuthology.orchestra.run.smithi042.stdout:5655 trunc from 0x75d251b to 0x98fa683 2024-09-18T00:59:41.177 INFO:teuthology.orchestra.run.smithi042.stdout:5658 trunc from 0x98fa683 to 0xc0843ad 2024-09-18T00:59:41.186 INFO:teuthology.orchestra.run.smithi042.stdout:5659 punch from 0x65bebff to 0x65c3312, (0x4713 bytes) 2024-09-18T00:59:41.186 INFO:teuthology.orchestra.run.smithi042.stdout:5660 read 0xb126f5a thru 0xb12c786 (0x582d bytes) 2024-09-18T00:59:41.187 INFO:teuthology.orchestra.run.smithi042.stdout:5661 read 0xd3ac9 thru 0xd410e (0x646 bytes) 2024-09-18T00:59:41.189 INFO:teuthology.orchestra.run.smithi042.stdout:5663 writesame 0x18f87ee thru 0x1908534 data_size 0x1(0xfd47 bytes) 2024-09-18T00:59:41.229 INFO:teuthology.orchestra.run.smithi042.stdout:5664 read 0x4465f thru 0x47c3b (0x35dd bytes) 2024-09-18T00:59:41.230 INFO:teuthology.orchestra.run.smithi042.stdout:5665 punch from 0x687c3d2 to 0x6884e3e, (0x8a6c bytes) 2024-09-18T00:59:41.230 INFO:teuthology.orchestra.run.smithi042.stdout:5666 write 0x1b2d9ae thru 0x1b3bae0 (0xe133 bytes) 2024-09-18T00:59:41.238 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:41.237+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:41.241 INFO:teuthology.orchestra.run.smithi042.stdout:5667 read 0xa0b0fdf thru 0xa0b1d38 (0xd5a bytes) 2024-09-18T00:59:41.241 INFO:teuthology.orchestra.run.smithi042.stdout:5669 read 0x715851b thru 0x716282c (0xa312 bytes) 2024-09-18T00:59:41.241 INFO:teuthology.orchestra.run.smithi042.stdout:5670 writesame 0x34a1256 thru 0x34a56c8 data_size 0x3b(0x4473 bytes) 2024-09-18T00:59:41.248 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:59:41.249 INFO:teuthology.orchestra.run.smithi042.stdout:5671 punch from 0x54f19af to 0x54f8c7f, (0x72d0 bytes) 2024-09-18T00:59:41.249 INFO:teuthology.orchestra.run.smithi042.stdout:5672 writesame 0x83cf459 thru 0x83d6a65 data_size 0x1(0x760d bytes) 2024-09-18T00:59:41.251 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:41.251 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:41.251 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:41.251 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:41.251 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:41.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:41.274 INFO:teuthology.orchestra.run.smithi042.stdout:5673 write 0x98f12ee thru 0x98fa233 (0x8f46 bytes) 2024-09-18T00:59:41.284 INFO:teuthology.orchestra.run.smithi042.stdout:5674 write 0x4878eaa thru 0x48884d0 (0xf627 bytes) 2024-09-18T00:59:41.298 INFO:teuthology.orchestra.run.smithi042.stdout:5675 writesame 0x67c9057 thru 0x67cbe48 data_size 0x1(0x2df2 bytes) 2024-09-18T00:59:41.318 INFO:teuthology.orchestra.run.smithi042.stdout:5676 trunc from 0xc0843ad to 0x7d5a18c 2024-09-18T00:59:41.338 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:41.337+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:41.339 INFO:teuthology.orchestra.run.smithi042.stdout:5677 read 0xa67d29 thru 0xa74837 (0xcb0f bytes) 2024-09-18T00:59:41.341 INFO:teuthology.orchestra.run.smithi042.stdout:5679 read 0x318ad76 thru 0x318f07d (0x4308 bytes) 2024-09-18T00:59:41.342 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:41.342 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:41.342 INFO:teuthology.orchestra.run.smithi042.stdout:5682 read 0x3e0475c thru 0x3e115d6 (0xce7b bytes) 2024-09-18T00:59:41.375 INFO:teuthology.orchestra.run.smithi042.stdout:5683 writesame 0x7f0efcf thru 0x7f18ead data_size 0x3(0x9edf bytes) 2024-09-18T00:59:41.381 INFO:teuthology.orchestra.run.smithi042.stdout:5685 trunc from 0x7f18eae to 0x268300d 2024-09-18T00:59:41.402 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.323513101, 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:59:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:59:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:59:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.32258351, 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:59:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:59:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.32256945, 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:59:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.321278875, 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:59:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.321258403, 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:59:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:59:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.31846394999999, 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:59:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.318448766, 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:59:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.31842582, 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:59:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.314248181, 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:59:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.314226964, 2024-09-18T00:59:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.314205581, 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:59:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.310305727, 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:59:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:59:41.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.310291526, 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.310273489, 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:41.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.31025942300001, 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:41.471 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.306234072, 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:41.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.30622006, 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.476 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:59:41.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.306206096, 2024-09-18T00:59:41.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:59:41.484 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.544348142, 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:59:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:59:41.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.418871770999999, 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:59:41.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:59:41.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:41.495 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:41.497 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:41.498 INFO:teuthology.orchestra.run.smithi042.stdout:5686 write 0xaed8964 thru 0xaee1bdb (0x9278 bytes) 2024-09-18T00:59:41.498 INFO:teuthology.orchestra.run.smithi042.stdout:5687 read 0xa292fa thru 0xa34e16 (0xbb1d bytes) 2024-09-18T00:59:41.498 INFO:teuthology.orchestra.run.smithi042.stdout:5688 read 0x3fad014 thru 0x3fbcf28 (0xff15 bytes) 2024-09-18T00:59:41.499 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:41.499 INFO:teuthology.orchestra.run.smithi042.stdout:5690 write 0x70657c7 thru 0x70723ba (0xcbf4 bytes) 2024-09-18T00:59:41.499 INFO:teuthology.orchestra.run.smithi042.stdout:5691 write 0xe182cfc thru 0xe18ab9b (0x7ea0 bytes) 2024-09-18T00:59:41.499 INFO:teuthology.orchestra.run.smithi042.stdout:5692 punch from 0x3b021ba to 0x3b06f91, (0x4dd7 bytes) 2024-09-18T00:59:41.499 INFO:teuthology.orchestra.run.smithi042.stdout:5693 trunc from 0xe18ab9c to 0x1394650 2024-09-18T00:59:41.499 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:41.497+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:41.536 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:41.536 INFO:teuthology.orchestra.run.smithi042.stdout:5695 punch from 0xf2c1d2 to 0xf2d1a7, (0xfd5 bytes) 2024-09-18T00:59:41.536 INFO:teuthology.orchestra.run.smithi042.stdout:5696 read 0x67195c thru 0x677eb1 (0x6556 bytes) 2024-09-18T00:59:41.537 INFO:teuthology.orchestra.run.smithi042.stdout:5698 read 0x325c3f thru 0x32cfbd (0x737f bytes) 2024-09-18T00:59:41.563 INFO:teuthology.orchestra.run.smithi042.stdout:5700 writesame 0xb8261f0 thru 0xb82bf02 data_size 0x1(0x5d13 bytes) 2024-09-18T00:59:41.563 INFO:teuthology.orchestra.run.smithi042.stdout:5701 writesame 0xb54ba81 thru 0xb55405e data_size 0x73(0x85de bytes) 2024-09-18T00:59:41.585 INFO:teuthology.orchestra.run.smithi042.stdout:5702 trunc from 0xb82bf03 to 0x13377b6 2024-09-18T00:59:41.598 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:41.597+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:41.607 INFO:teuthology.orchestra.run.smithi042.stdout:5703 read 0x4b6e50 thru 0x4bf721 (0x88d2 bytes) 2024-09-18T00:59:41.608 INFO:teuthology.orchestra.run.smithi042.stdout:5704 read 0x12f1734 thru 0x12ff454 (0xdd21 bytes) 2024-09-18T00:59:41.638 INFO:teuthology.orchestra.run.smithi042.stdout:5705 write 0xe1a2129 thru 0xe1a58ed (0x37c5 bytes) 2024-09-18T00:59:41.645 INFO:teuthology.orchestra.run.smithi042.stdout:5706 write 0x363a735 thru 0x3641fab (0x7877 bytes) 2024-09-18T00:59:41.656 INFO:teuthology.orchestra.run.smithi042.stdout:5707 punch from 0xdf39798 to 0xdf48337, (0xeb9f bytes) 2024-09-18T00:59:41.656 INFO:teuthology.orchestra.run.smithi042.stdout:5709 read 0xa6603cd thru 0xa6608c9 (0x4fd bytes) 2024-09-18T00:59:41.656 INFO:teuthology.orchestra.run.smithi042.stdout:5710 writesame 0xe4c4c2 thru 0xe57e43 data_size 0xf(0xb982 bytes) 2024-09-18T00:59:41.668 INFO:teuthology.orchestra.run.smithi042.stdout:5711 read 0xa48fbe4 thru 0xa49a169 (0xa586 bytes) 2024-09-18T00:59:41.668 INFO:teuthology.orchestra.run.smithi042.stdout:5712 read 0x9e729e8 thru 0x9e7b154 (0x876d bytes) 2024-09-18T00:59:41.668 INFO:teuthology.orchestra.run.smithi042.stdout:5713 write 0x918fe90 thru 0x919c0c2 (0xc233 bytes) 2024-09-18T00:59:41.668 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:41.668 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:41.668 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:41.668 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:41.679 INFO:teuthology.orchestra.run.smithi042.stdout:5714 flatten 2024-09-18T00:59:41.682 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:41.700 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:41.697+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:41.719 INFO:teuthology.orchestra.run.smithi042.stdout:5717 writesame 0x5ae62c9 thru 0x5af1963 data_size 0x1(0xb69b bytes) 2024-09-18T00:59:41.753 INFO:teuthology.orchestra.run.smithi042.stdout:5718 read 0x7029bec thru 0x7039356 (0xf76b bytes) 2024-09-18T00:59:41.753 INFO:teuthology.orchestra.run.smithi042.stdout:5719 writesame 0xb7ba3e1 thru 0xb7c796f data_size 0x1(0xd58f bytes) 2024-09-18T00:59:41.791 INFO:teuthology.orchestra.run.smithi042.stdout:5720 write 0x53c2ff thru 0x53e978 (0x267a bytes) 2024-09-18T00:59:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:5721 punch from 0x2db66c3 to 0x2dc2877, (0xc1b4 bytes) 2024-09-18T00:59:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:5723 read 0x3ad2b1d thru 0x3adf796 (0xcc7a bytes) 2024-09-18T00:59:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:5726 read 0x2cb924e thru 0x2cc3cd7 (0xaa8a bytes) 2024-09-18T00:59:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:5727 punch from 0x66ee722 to 0x66eefb2, (0x890 bytes) 2024-09-18T00:59:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:5728 read 0xa8c1409 thru 0xa8c1680 (0x278 bytes) 2024-09-18T00:59:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:5729 write 0x554831d thru 0x554d566 (0x524a bytes) 2024-09-18T00:59:41.799 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:41.797+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:41.803 INFO:teuthology.orchestra.run.smithi042.stdout:5730 read 0x2c3c41 thru 0x2d2a8c (0xee4c bytes) 2024-09-18T00:59:41.804 INFO:teuthology.orchestra.run.smithi042.stdout:5731 read 0xa95c800 thru 0xa963f05 (0x7706 bytes) 2024-09-18T00:59:41.805 INFO:teuthology.orchestra.run.smithi042.stdout:5732 write 0x89affd9 thru 0x89b8a3a (0x8a62 bytes) 2024-09-18T00:59:41.842 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:41.842 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:41.842 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:41.843 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:41.843 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:41.847 INFO:teuthology.orchestra.run.smithi042.stdout:5733 trunc from 0xe1a58ee to 0xfda2d5 2024-09-18T00:59:41.860 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:41.895 INFO:teuthology.orchestra.run.smithi042.stdout:5736 read 0xc3ce00 thru 0xc478b8 (0xaab9 bytes) 2024-09-18T00:59:41.900 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:41.901+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:41.918 INFO:teuthology.orchestra.run.smithi042.stdout:5737 trunc from 0xfda2d5 to 0x9cb0ee8 2024-09-18T00:59:41.930 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:41.937 INFO:teuthology.orchestra.run.smithi042.stdout:5739 trunc from 0x9cb0ee8 to 0x705f954 2024-09-18T00:59:41.951 INFO:teuthology.orchestra.run.smithi042.stdout:5740 writesame 0x353021a thru 0x353eaee data_size 0x5b(0xe8d5 bytes) 2024-09-18T00:59:41.966 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:41.966 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:41.967 INFO:teuthology.orchestra.run.smithi042.stdout:5743 read 0x15f7696 thru 0x15fc12e (0x4a99 bytes) 2024-09-18T00:59:41.967 INFO:teuthology.orchestra.run.smithi042.stdout:5745 write 0x4a27731 thru 0x4a32e8f (0xb75f bytes) 2024-09-18T00:59:42.001 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:42.001+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:42.005 INFO:teuthology.orchestra.run.smithi042.stdout:5746 write 0xb53dc1e thru 0xb5429fb (0x4dde bytes) 2024-09-18T00:59:42.036 INFO:teuthology.orchestra.run.smithi042.stdout:5747 trunc from 0xb5429fc to 0x1992771 2024-09-18T00:59:42.042 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.042 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98052 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4179968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.600291968999997, 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131360583, 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98052 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440037+0000", 2024-09-18T00:59:42.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440040+0000", 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7159999999999999e-06 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440048+0000", 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7169999999999997e-06 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440056+0000", 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1860000000000003e-06 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.570989+0000", 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13093384999999999 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571175+0000", 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018504799999999999 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571397+0000", 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022245899999999999 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98053 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4466688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.600264099999997, 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13182195299999999, 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.047 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98053 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440065+0000", 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440068+0000", 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6699999999999998e-06 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440076+0000", 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7069999999999998e-06 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440083+0000", 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9929999999999998e-06 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571475+0000", 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13139158100000001 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571657+0000", 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018247099999999999 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571886+0000", 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022892400000000001 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98054 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2113536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:42.051 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.600236240000001, 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13228911600000001, 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98054 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440093+0000", 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440096+0000", 2024-09-18T00:59:42.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440105+0000", 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7739999999999993e-06 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440112+0000", 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2130000000000002e-06 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571970+0000", 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13185812199999999 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572159+0000", 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000188665 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572381+0000", 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022273200000000001 2024-09-18T00:59:42.055 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98055 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2400256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.600206059999998, 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273966600000001, 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.056 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98055 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440123+0000", 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440126+0000", 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440137+0000", 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0543000000000001e-05 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440144+0000", 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572457+0000", 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132312807 2024-09-18T00:59:42.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572636+0000", 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882299999999999 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572862+0000", 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022645200000000001 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98056 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2686976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.600175483000001, 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13308801200000001, 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98056 2024-09-18T00:59:42.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440154+0000", 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5300000000000004e-07 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440156+0000", 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.903e-06 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440165+0000", 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7600000000000008e-06 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440174+0000", 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6969999999999999e-06 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572960+0000", 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132785984 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573087+0000", 2024-09-18T00:59:42.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012728699999999999 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573241+0000", 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153728 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98057 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2973696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:42.065 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.60014589, 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13357376900000001, 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98057 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440183+0000", 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440186+0000", 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8700000000000001e-06 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440195+0000", 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7099999999999996e-06 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440202+0000", 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2429999999999999e-06 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573338+0000", 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13313578400000001 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573526+0000", 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018780399999999999 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573756+0000", 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230755 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98058 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.600118006999999, 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13387749400000001, 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98058 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440627+0000", 2024-09-18T00:59:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041660699999999999 2024-09-18T00:59:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440634+0000", 2024-09-18T00:59:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.596e-06 2024-09-18T00:59:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440650+0000", 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6164e-05 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440660+0000", 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.023e-05 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573833+0000", 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13317317000000001 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573945+0000", 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111731 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574088+0000", 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000142996 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98059 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3547136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.599650660000002, 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13391484000000001, 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98059 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441760+0000", 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001082185 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441780+0000", 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0463e-05 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441829+0000", 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8763000000000001e-05 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441849+0000", 2024-09-18T00:59:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9423e-05 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574173+0000", 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13232460400000001 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574367+0000", 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019403000000000001 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574593+0000", 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000225372 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98060 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3833856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.598448578999999, 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13320725899999999, 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98060 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441881+0000", 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6160000000000001e-06 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441889+0000", 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8439999999999998e-06 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441909+0000", 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9856000000000001e-05 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441925+0000", 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6039999999999999e-05 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574669+0000", 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13274412799999999 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574860+0000", 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019038599999999999 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575087+0000", 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022738899999999999 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98061 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4120576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.598381822999997, 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13362664099999999, 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98061 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441948+0000", 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441954+0000", 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3099999999999997e-06 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441981+0000", 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7095999999999999e-05 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441998+0000", 2024-09-18T00:59:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6237000000000001e-05 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575165+0000", 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13316721200000001 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575355+0000", 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000190218 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575573+0000", 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021824799999999999 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98062 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4407296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.598309854, 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13404179099999999, 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98062 2024-09-18T00:59:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445789+0000", 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037705569999999999 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445805+0000", 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.558e-05 2024-09-18T00:59:42.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445846+0000", 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445871+0000", 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5072999999999999e-05 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575651+0000", 2024-09-18T00:59:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12977999600000001 2024-09-18T00:59:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575835+0000", 2024-09-18T00:59:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000183931 2024-09-18T00:59:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576060+0000", 2024-09-18T00:59:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022558800000000001 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98064 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2115584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.594345414999999, 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13103894799999999, 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98064 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445985+0000", 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445991+0000", 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446014+0000", 2024-09-18T00:59:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3056000000000001e-05 2024-09-18T00:59:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446030+0000", 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5662999999999999e-05 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576622+0000", 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13059273900000001 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576805+0000", 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018304899999999999 2024-09-18T00:59:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577022+0000", 2024-09-18T00:59:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216481 2024-09-18T00:59:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98065 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2402304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.594278299000003, 2024-09-18T00:59:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131457463, 2024-09-18T00:59:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98065 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446051+0000", 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1260000000000001e-06 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446057+0000", 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2040000000000002e-06 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446077+0000", 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446092+0000", 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5124e-05 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577098+0000", 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13100595100000001 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577284+0000", 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000186152 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577508+0000", 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022314000000000001 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98066 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2689024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.594216093, 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13189522100000001, 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98066 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446113+0000", 2024-09-18T00:59:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0240000000000001e-06 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446120+0000", 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446137+0000", 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7479e-05 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446152+0000", 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4633e-05 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577587+0000", 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13143572100000001 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577786+0000", 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019872999999999999 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578008+0000", 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022141399999999999 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98067 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2975744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.59415843, 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132319567, 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98067 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446171+0000", 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446178+0000", 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699999999999997e-06 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446196+0000", 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7989999999999999e-05 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446213+0000", 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7493999999999999e-05 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578083+0000", 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13186995000000001 2024-09-18T00:59:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578266+0000", 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018321099999999999 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578490+0000", 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022316600000000001 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98068 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3262464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.594097523999999, 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273661, 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98068 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446232+0000", 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.187e-06 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446238+0000", 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446257+0000", 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446272+0000", 2024-09-18T00:59:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5532999999999998e-05 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578569+0000", 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132296631 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578749+0000", 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017978300000000001 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578967+0000", 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218736 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98069 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3549184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.594037851000003, 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13311771, 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98069 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446292+0000", 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446298+0000", 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4470000000000003e-06 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446340+0000", 2024-09-18T00:59:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1705999999999997e-05 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446358+0000", 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7873999999999999e-05 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579072+0000", 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13271396399999999 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579213+0000", 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014133900000000001 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579408+0000", 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019492700000000001 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98070 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3835904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.593944407999999, 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133517145, 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98070 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446385+0000", 2024-09-18T00:59:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446393+0000", 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2629999999999997e-06 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446413+0000", 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9930000000000001e-05 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446428+0000", 2024-09-18T00:59:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5336e-05 2024-09-18T00:59:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579494+0000", 2024-09-18T00:59:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13306637399999999 2024-09-18T00:59:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579682+0000", 2024-09-18T00:59:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018740900000000001 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579901+0000", 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021946300000000001 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98071 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.593881242000002, 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133931615, 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98071 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449904+0000", 2024-09-18T00:59:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003456822 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449945+0000", 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0896000000000003e-05 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449971+0000", 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5996999999999999e-05 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449986+0000", 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.49e-05 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579978+0000", 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.129992723 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580157+0000", 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179006 2024-09-18T00:59:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580379+0000", 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022127099999999999 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98073 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4696064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.590272114999998, 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13113503100000001, 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98073 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450057+0000", 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450060+0000", 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5329999999999999e-06 2024-09-18T00:59:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450072+0000", 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1919999999999999e-05 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450080+0000", 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9500000000000001e-06 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580951+0000", 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.130871128 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581057+0000", 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010501399999999999 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581191+0000", 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013479900000000001 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:42.137 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:42.137 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:42.139 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:42.140 INFO:teuthology.orchestra.run.smithi042.stdout:5748 punch from 0x7c978c to 0x7cf96d, (0x61e1 bytes) 2024-09-18T00:59:42.140 INFO:teuthology.orchestra.run.smithi042.stdout:5749 write 0xc76c7a3 thru 0xc775016 (0x8874 bytes) 2024-09-18T00:59:42.140 INFO:teuthology.orchestra.run.smithi042.stdout:5750 write 0xe6356b4 thru 0xe638855 (0x31a2 bytes) 2024-09-18T00:59:42.140 INFO:teuthology.orchestra.run.smithi042.stdout:5751 write 0x2ebe29c thru 0x2ec8ee3 (0xac48 bytes) 2024-09-18T00:59:42.140 INFO:teuthology.orchestra.run.smithi042.stdout:5752 writesame 0x606711f thru 0x60693a6 data_size 0x28(0x2288 bytes) 2024-09-18T00:59:42.140 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:42.137+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:42.146 INFO:teuthology.orchestra.run.smithi042.stdout:5754 punch from 0x2a1b0ec to 0x2a215a6, (0x64ba bytes) 2024-09-18T00:59:42.146 INFO:teuthology.orchestra.run.smithi042.stdout:5756 write 0x214e41c thru 0x215b3b3 (0xcf98 bytes) 2024-09-18T00:59:42.158 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:42.158 INFO:teuthology.orchestra.run.smithi042.stdout:5759 write 0xcbe66a2 thru 0xcbeb419 (0x4d78 bytes) 2024-09-18T00:59:42.164 INFO:teuthology.orchestra.run.smithi042.stdout:5760 writesame 0xe55111e thru 0xe557409 data_size 0x1a(0x62ec bytes) 2024-09-18T00:59:42.170 INFO:teuthology.orchestra.run.smithi042.stdout:5762 writesame 0x4e4cf51 thru 0x4e5a8e5 data_size 0x9(0xd995 bytes) 2024-09-18T00:59:42.181 INFO:teuthology.orchestra.run.smithi042.stdout:5764 read 0x8ec1af0 thru 0x8ec93e7 (0x78f8 bytes) 2024-09-18T00:59:42.202 INFO:teuthology.orchestra.run.smithi042.stdout:5765 trunc from 0xe638856 to 0x39c3ab9 2024-09-18T00:59:42.235 INFO:teuthology.orchestra.run.smithi042.stdout:5766 write 0x39cfe05 thru 0x39da3f1 (0xa5ed bytes) 2024-09-18T00:59:42.239 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:42.237+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:42.246 INFO:teuthology.orchestra.run.smithi042.stdout:5768 read 0x1d1aa9f thru 0x1d2977a (0xecdc bytes) 2024-09-18T00:59:42.250 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:59:42.270 INFO:teuthology.orchestra.run.smithi042.stdout:5769 write 0xbc16459 thru 0xbc17bcb (0x1773 bytes) 2024-09-18T00:59:42.274 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:42.285 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:42.285 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:42.285 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:42.285 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:42.287 INFO:teuthology.orchestra.run.smithi042.stdout:5772 write 0xeba9096 thru 0xebb6b5f (0xdaca bytes) 2024-09-18T00:59:42.296 INFO:teuthology.orchestra.run.smithi042.stdout:5773 write 0x5679da2 thru 0x5687052 (0xd2b1 bytes) 2024-09-18T00:59:42.304 INFO:teuthology.orchestra.run.smithi042.stdout:5774 read 0xbe3ade8 thru 0xbe493b3 (0xe5cc bytes) 2024-09-18T00:59:42.304 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:42.304 INFO:teuthology.orchestra.run.smithi042.stdout:5776 punch from 0xd0bc7bd to 0xd0be7f5, (0x2038 bytes) 2024-09-18T00:59:42.304 INFO:teuthology.orchestra.run.smithi042.stdout:5777 write 0xbdcfba1 thru 0xbdd1872 (0x1cd2 bytes) 2024-09-18T00:59:42.305 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:42.308 INFO:teuthology.orchestra.run.smithi042.stdout:5778 write 0x34b2070 thru 0x34b7eb9 (0x5e4a bytes) 2024-09-18T00:59:42.313 INFO:teuthology.orchestra.run.smithi042.stdout:5779 read 0xd7b6eb8 thru 0xd7c497a (0xdac3 bytes) 2024-09-18T00:59:42.316 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:42.316 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:42.333 INFO:teuthology.orchestra.run.smithi042.stdout:5780 trunc from 0xebb6b60 to 0xcef391 2024-09-18T00:59:42.334 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:42.334 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:42.339 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:42.337+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:42.353 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:42.355 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:42.355 INFO:teuthology.orchestra.run.smithi042.stdout:5783 punch from 0x3cdd79 to 0x3d66ed, (0x8974 bytes) 2024-09-18T00:59:42.355 INFO:teuthology.orchestra.run.smithi042.stdout:5787 read 0x88d65b thru 0x89cbf1 (0xf597 bytes) 2024-09-18T00:59:42.372 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:42.373 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:42.374 INFO:teuthology.orchestra.run.smithi042.stdout:5788 trunc from 0xcef391 to 0xd409eee 2024-09-18T00:59:42.381 INFO:teuthology.orchestra.run.smithi042.stdout:5789 punch from 0xbfa3cbe to 0xbfb31ca, (0xf50c bytes) 2024-09-18T00:59:42.381 INFO:teuthology.orchestra.run.smithi042.stdout:5790 write 0x8a45e1d thru 0x8a4fb60 (0x9d44 bytes) 2024-09-18T00:59:42.387 INFO:teuthology.orchestra.run.smithi042.stdout:5791 punch from 0x3186a0e to 0x318a19a, (0x378c bytes) 2024-09-18T00:59:42.387 INFO:teuthology.orchestra.run.smithi042.stdout:5793 write 0x6db3ba7 thru 0x6db7ba8 (0x4002 bytes) 2024-09-18T00:59:42.393 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:42.393 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:42.400 INFO:teuthology.orchestra.run.smithi042.stdout:5794 trunc from 0xd409eee to 0x7e8eca3 2024-09-18T00:59:42.414 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:42.414 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:42.415 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:42.415 INFO:teuthology.orchestra.run.smithi042.stdout:5797 write 0x4e59b15 thru 0x4e5b0a4 (0x1590 bytes) 2024-09-18T00:59:42.433 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:42.433 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:42.439 INFO:teuthology.orchestra.run.smithi042.stdout:5799 write 0xed30f69 thru 0xed3adc7 (0x9e5f bytes) 2024-09-18T00:59:42.440 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:42.437+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:42.445 INFO:teuthology.orchestra.run.smithi042.stdout:5801 read 0x7e7221c thru 0x7e75356 (0x313b bytes) 2024-09-18T00:59:42.455 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:42.455 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:42.461 INFO:teuthology.orchestra.run.smithi042.stdout:5803 trunc from 0xed3adc8 to 0x51c8d7c 2024-09-18T00:59:42.477 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:42.477 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:42.492 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:42.495 INFO:teuthology.orchestra.run.smithi042.stdout:5805 writesame 0x96ee6b2 thru 0x96ee88a data_size 0x1(0x1d9 bytes) 2024-09-18T00:59:42.495 INFO:teuthology.orchestra.run.smithi042.stdout:5806 punch from 0x2ed681d to 0x2ee3651, (0xce34 bytes) 2024-09-18T00:59:42.495 INFO:teuthology.orchestra.run.smithi042.stdout:5811 read 0x44e576a thru 0x44ea120 (0x49b7 bytes) 2024-09-18T00:59:42.495 INFO:teuthology.orchestra.run.smithi042.stdout:5812 punch from 0xb25dbd to 0xb28a46, (0x2c89 bytes) 2024-09-18T00:59:42.495 INFO:teuthology.orchestra.run.smithi042.stdout:5813 read 0xdc4fd1 thru 0xdc9748 (0x4778 bytes) 2024-09-18T00:59:42.506 INFO:teuthology.orchestra.run.smithi042.stdout:5814 write 0xadce34c thru 0xaddb83a (0xd4ef bytes) 2024-09-18T00:59:42.527 INFO:teuthology.orchestra.run.smithi042.stdout:5815 write 0xd907d74 thru 0xd912684 (0xa911 bytes) 2024-09-18T00:59:42.536 INFO:teuthology.orchestra.run.smithi042.stdout:5816 read 0x623f4ad thru 0x624a7b0 (0xb304 bytes) 2024-09-18T00:59:42.536 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:42.536 INFO:teuthology.orchestra.run.smithi042.stdout:5818 clone 43 order 22 su 524288 sc 10 2024-09-18T00:59:42.541 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:42.541+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:42.612 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:42.613 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:42.626 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:42.642 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:42.641+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:42.743 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:42.741+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:42.789 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:42.789 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:42.803 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:42.844 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:42.841+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:42.926 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:42.926 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:42.940 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:42.984 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:42.985+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:43.045 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:43.045+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:43.061 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:43.061 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:43.075 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:43.145 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:43.141+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:43.240 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:43.240 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:43.247 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:43.245+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:43.255 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:43.347 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:43.345+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:43.382 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:43.382 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:43.396 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:43.484 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:43.485+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:43.550 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:43.549+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:43.561 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:43.561 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:43.576 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:43.579 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-18T00:59:43.664 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:43.665+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:43.745 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:43.746 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:43.751 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:43.749+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:43.760 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:43.852 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:43.853+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:43.918 INFO:teuthology.orchestra.run.smithi042.stdout:5820 read 0x2677e84 thru 0x267b201 (0x337e bytes) 2024-09-18T00:59:43.926 INFO:teuthology.orchestra.run.smithi042.stdout:5821 read 0x9ac468e thru 0x9acc998 (0x830b bytes) 2024-09-18T00:59:43.927 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:43.927 INFO:teuthology.orchestra.run.smithi042.stdout:5826 punch from 0x69179cb to 0x69254ac, (0xdae1 bytes) 2024-09-18T00:59:43.927 INFO:teuthology.orchestra.run.smithi042.stdout:5827 read 0x7c2da9f thru 0x7c34a2f (0x6f91 bytes) 2024-09-18T00:59:43.927 INFO:teuthology.orchestra.run.smithi042.stdout:5828 read 0x708c33a thru 0x709b0db (0xeda2 bytes) 2024-09-18T00:59:43.928 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:43.928 INFO:teuthology.orchestra.run.smithi042.stdout:5830 writesame 0x9de89ba thru 0x9df7ab7 data_size 0x6d(0xf0fe bytes) 2024-09-18T00:59:43.934 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:43.934 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:43.941 INFO:teuthology.orchestra.run.smithi042.stdout:5833 read 0x3b11315 thru 0x3b19cf3 (0x89df bytes) 2024-09-18T00:59:43.941 INFO:teuthology.orchestra.run.smithi042.stdout:5834 write 0x11fd9c6 thru 0x120a1ef (0xc82a bytes) 2024-09-18T00:59:43.953 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:43.949+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:43.954 INFO:teuthology.orchestra.run.smithi042.stdout:5835 write 0x896cd2c thru 0x89786c0 (0xb995 bytes) 2024-09-18T00:59:43.955 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:43.961 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:43.961 INFO:teuthology.orchestra.run.smithi042.stdout:5839 writesame 0xd8018df thru 0xd80dade data_size 0x20(0xc200 bytes) 2024-09-18T00:59:43.967 INFO:teuthology.orchestra.run.smithi042.stdout:5840 write 0x94de336 thru 0x94dfa02 (0x16cd bytes) 2024-09-18T00:59:43.972 INFO:teuthology.orchestra.run.smithi042.stdout:5841 write 0xa3e7f82 thru 0xa3f42c9 (0xc348 bytes) 2024-09-18T00:59:43.990 INFO:teuthology.orchestra.run.smithi042.stdout:5842 write 0xe8e334c thru 0xe8e55d8 (0x228d bytes) 2024-09-18T00:59:43.995 INFO:teuthology.orchestra.run.smithi042.stdout:5843 punch from 0x984fa4c to 0x985713d, (0x76f1 bytes) 2024-09-18T00:59:43.997 INFO:teuthology.orchestra.run.smithi042.stdout:5844 writesame 0xd53055 thru 0xd57a09 data_size 0x1(0x49b5 bytes) 2024-09-18T00:59:44.008 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6] out_osds: [2, 3, 0, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:59:44.008 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:59:44.008 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T00:59:44.009 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 1.000000 2024-09-18T00:59:44.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-18T00:59:44.018 INFO:teuthology.orchestra.run.smithi042.stdout:5845 punch from 0x604e5f5 to 0x6055b7e, (0x7589 bytes) 2024-09-18T00:59:44.018 INFO:teuthology.orchestra.run.smithi042.stdout:5846 read 0xe677df9 thru 0xe67cec3 (0x50cb bytes) 2024-09-18T00:59:44.018 INFO:teuthology.orchestra.run.smithi042.stdout:5847 write 0x4a80ca9 thru 0x4a82cc8 (0x2020 bytes) 2024-09-18T00:59:44.038 INFO:teuthology.orchestra.run.smithi042.stdout:5848 trunc from 0xe8e55d9 to 0x885b503 2024-09-18T00:59:44.053 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:44.049+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:44.064 INFO:teuthology.orchestra.run.smithi042.stdout:5849 read 0x757a67a thru 0x7586708 (0xc08f bytes) 2024-09-18T00:59:44.064 INFO:teuthology.orchestra.run.smithi042.stdout:5851 read 0x392d41b thru 0x3933f97 (0x6b7d bytes) 2024-09-18T00:59:44.065 INFO:teuthology.orchestra.run.smithi042.stdout:5852 write 0x563736f thru 0x5640357 (0x8fe9 bytes) 2024-09-18T00:59:44.070 INFO:teuthology.orchestra.run.smithi042.stdout:5853 write 0x176c2b4 thru 0x177af3d (0xec8a bytes) 2024-09-18T00:59:44.081 INFO:teuthology.orchestra.run.smithi042.stdout:5854 trunc from 0x885b503 to 0x3acc2e9 2024-09-18T00:59:44.118 INFO:teuthology.orchestra.run.smithi042.stdout:5855 read 0x3021b5d thru 0x3026464 (0x4908 bytes) 2024-09-18T00:59:44.136 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:44.136 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:44.148 INFO:teuthology.orchestra.run.smithi042.stdout:5857 writesame 0xbb376c9 thru 0xbb3adad data_size 0x17(0x36e5 bytes) 2024-09-18T00:59:44.148 INFO:teuthology.orchestra.run.smithi042.stdout:5859 read 0x140a5e4 thru 0x140f39a (0x4db7 bytes) 2024-09-18T00:59:44.149 INFO:teuthology.orchestra.run.smithi042.stdout:5860 read 0x1f2340b thru 0x1f2978a (0x6380 bytes) 2024-09-18T00:59:44.150 INFO:teuthology.orchestra.run.smithi042.stdout:5862 punch from 0x6921997 to 0x6921bd8, (0x241 bytes) 2024-09-18T00:59:44.153 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:44.155 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:44.153+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:44.160 INFO:teuthology.orchestra.run.smithi042.stdout:5865 trunc from 0xbb3adae to 0x6234d4a 2024-09-18T00:59:44.169 INFO:teuthology.orchestra.run.smithi042.stdout:5866 punch from 0x245370d to 0x245c8d6, (0x91c9 bytes) 2024-09-18T00:59:44.169 INFO:teuthology.orchestra.run.smithi042.stdout:5867 punch from 0x13e968d to 0x13eebd5, (0x5548 bytes) 2024-09-18T00:59:44.169 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:44.169 INFO:teuthology.orchestra.run.smithi042.stdout:5874 read 0x44c3bf2 thru 0x44c8e93 (0x52a2 bytes) 2024-09-18T00:59:44.169 INFO:teuthology.orchestra.run.smithi042.stdout:5875 read 0x43fec6d thru 0x4401d85 (0x3119 bytes) 2024-09-18T00:59:44.170 INFO:teuthology.orchestra.run.smithi042.stdout:5876 write 0x190ad1 thru 0x19d376 (0xc8a6 bytes) 2024-09-18T00:59:44.180 INFO:teuthology.orchestra.run.smithi042.stdout:5877 read 0x369ed0c thru 0x36a470d (0x5a02 bytes) 2024-09-18T00:59:44.181 INFO:teuthology.orchestra.run.smithi042.stdout:5879 punch from 0xdc36fd to 0xdc3ee4, (0x7e7 bytes) 2024-09-18T00:59:44.200 INFO:teuthology.orchestra.run.smithi042.stdout:5881 writesame 0xbccf205 thru 0xbcdd665 data_size 0xb(0xe461 bytes) 2024-09-18T00:59:44.200 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:44.200 INFO:teuthology.orchestra.run.smithi042.stdout:5883 read 0x214ef6e thru 0x215935b (0xa3ee bytes) 2024-09-18T00:59:44.201 INFO:teuthology.orchestra.run.smithi042.stdout:5884 read 0x88ca148 thru 0x88d71f5 (0xd0ae bytes) 2024-09-18T00:59:44.201 INFO:teuthology.orchestra.run.smithi042.stdout:5887 read 0xaec320a thru 0xaec50cd (0x1ec4 bytes) 2024-09-18T00:59:44.201 INFO:teuthology.orchestra.run.smithi042.stdout:5888 punch from 0xae4a333 to 0xae52856, (0x8523 bytes) 2024-09-18T00:59:44.201 INFO:teuthology.orchestra.run.smithi042.stdout:5889 read 0xb191c75 thru 0xb194364 (0x26f0 bytes) 2024-09-18T00:59:44.201 INFO:teuthology.orchestra.run.smithi042.stdout:5892 read 0x2db0422 thru 0x2db2f3c (0x2b1b bytes) 2024-09-18T00:59:44.201 INFO:teuthology.orchestra.run.smithi042.stdout:5897 write 0x8811471 thru 0x88128ea (0x147a bytes) 2024-09-18T00:59:44.205 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:44.206 INFO:teuthology.orchestra.run.smithi042.stdout:5899 write 0x44cbb3c thru 0x44d5625 (0x9aea bytes) 2024-09-18T00:59:44.219 INFO:teuthology.orchestra.run.smithi042.stdout:5900 write 0xda2c332 thru 0xda37e6b (0xbb3a bytes) 2024-09-18T00:59:44.224 INFO:teuthology.orchestra.run.smithi042.stdout:5901 write 0x790b783 thru 0x7910f51 (0x57cf bytes) 2024-09-18T00:59:44.227 INFO:teuthology.orchestra.run.smithi042.stdout:5902 read 0x3938f4 thru 0x3953ea (0x1af7 bytes) 2024-09-18T00:59:44.243 INFO:teuthology.orchestra.run.smithi042.stdout:5903 write 0xea982cb thru 0xeaa0bab (0x88e1 bytes) 2024-09-18T00:59:44.248 INFO:teuthology.orchestra.run.smithi042.stdout:5905 write 0x19305d9 thru 0x19370c0 (0x6ae8 bytes) 2024-09-18T00:59:44.250 INFO:teuthology.orchestra.run.smithi042.stdout:5907 read 0x1cb2c02 thru 0x1cb9178 (0x6577 bytes) 2024-09-18T00:59:44.251 INFO:teuthology.orchestra.run.smithi042.stdout:5908 write 0x159f4f9 thru 0x15a8192 (0x8c9a bytes) 2024-09-18T00:59:44.255 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:44.253+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:44.259 INFO:teuthology.orchestra.run.smithi042.stdout:5909 trunc from 0xeaa0bac to 0xcde008d 2024-09-18T00:59:44.274 INFO:teuthology.orchestra.run.smithi042.stdout:5910 writesame 0x3a0291e thru 0x3a03057 data_size 0xa(0x73a bytes) 2024-09-18T00:59:44.278 INFO:teuthology.orchestra.run.smithi042.stdout:5911 read 0x833eced thru 0x8345425 (0x6739 bytes) 2024-09-18T00:59:44.278 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:59:44.279 INFO:teuthology.orchestra.run.smithi042.stdout:5912 write 0x3b93912 thru 0x3b97342 (0x3a31 bytes) 2024-09-18T00:59:44.287 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:44.287 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:44.294 INFO:teuthology.orchestra.run.smithi042.stdout:5913 write 0xdbfeeee thru 0xdc024a4 (0x35b7 bytes) 2024-09-18T00:59:44.301 INFO:teuthology.orchestra.run.smithi042.stdout:5914 write 0x2201ee7 thru 0x2211500 (0xf61a bytes) 2024-09-18T00:59:44.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:44.310 INFO:teuthology.orchestra.run.smithi042.stdout:5915 read 0x9551789 thru 0x955b9b5 (0xa22d bytes) 2024-09-18T00:59:44.311 INFO:teuthology.orchestra.run.smithi042.stdout:5916 write 0x9680d74 thru 0x968ea74 (0xdd01 bytes) 2024-09-18T00:59:44.316 INFO:teuthology.orchestra.run.smithi042.stdout:5917 write 0xcf472b6 thru 0xcf48c06 (0x1951 bytes) 2024-09-18T00:59:44.323 INFO:teuthology.orchestra.run.smithi042.stdout:5920 write 0x71fe128 thru 0x71fe7d2 (0x6ab bytes) 2024-09-18T00:59:44.329 INFO:teuthology.orchestra.run.smithi042.stdout:5921 read 0x313ea60 thru 0x31488ac (0x9e4d bytes) 2024-09-18T00:59:44.330 INFO:teuthology.orchestra.run.smithi042.stdout:5922 write 0x2de2e57 thru 0x2df1b5e (0xed08 bytes) 2024-09-18T00:59:44.334 INFO:teuthology.orchestra.run.smithi042.stdout:5924 trunc from 0xdc024a5 to 0xde0c35f 2024-09-18T00:59:44.344 INFO:teuthology.orchestra.run.smithi042.stdout:5925 writesame 0xce7f10f thru 0xce86eac data_size 0x7(0x7d9e bytes) 2024-09-18T00:59:44.353 INFO:teuthology.orchestra.run.smithi042.stdout:5926 read 0xdb07f7c thru 0xdb0ee6b (0x6ef0 bytes) 2024-09-18T00:59:44.354 INFO:teuthology.orchestra.run.smithi042.stdout:5927 write 0x98774e5 thru 0x9885c12 (0xe72e bytes) 2024-09-18T00:59:44.356 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:44.353+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:44.362 INFO:teuthology.orchestra.run.smithi042.stdout:5928 read 0xd5a509e thru 0xd5a6310 (0x1273 bytes) 2024-09-18T00:59:44.374 INFO:teuthology.orchestra.run.smithi042.stdout:5929 write 0xe7b1780 thru 0xe7bccd2 (0xb553 bytes) 2024-09-18T00:59:44.377 INFO:teuthology.orchestra.run.smithi042.stdout:5930 read 0xe702c0b thru 0xe709270 (0x6666 bytes) 2024-09-18T00:59:44.377 INFO:teuthology.orchestra.run.smithi042.stdout:5931 read 0x928c69b thru 0x929024c (0x3bb2 bytes) 2024-09-18T00:59:44.377 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:44.378 INFO:teuthology.orchestra.run.smithi042.stdout:5935 read 0x9d6772e thru 0x9d6f0cf (0x79a2 bytes) 2024-09-18T00:59:44.378 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:44.379 INFO:teuthology.orchestra.run.smithi042.stdout:5937 read 0x57a14e6 thru 0x57ab343 (0x9e5e bytes) 2024-09-18T00:59:44.379 INFO:teuthology.orchestra.run.smithi042.stdout:5939 writesame 0xdd5d1d3 thru 0xdd6c1d4 data_size 0x1f(0xf002 bytes) 2024-09-18T00:59:44.384 INFO:teuthology.orchestra.run.smithi042.stdout:5940 writesame 0x641e9f7 thru 0x6421b99 data_size 0x1(0x31a3 bytes) 2024-09-18T00:59:44.399 INFO:teuthology.orchestra.run.smithi042.stdout:5941 write 0x4b70ab1 thru 0x4b71ff1 (0x1541 bytes) 2024-09-18T00:59:44.401 INFO:teuthology.orchestra.run.smithi042.stdout:5942 punch from 0x6094801 to 0x609d62b, (0x8e2a bytes) 2024-09-18T00:59:44.401 INFO:teuthology.orchestra.run.smithi042.stdout:5943 read 0x39643fe thru 0x396f49b (0xb09e bytes) 2024-09-18T00:59:44.402 INFO:teuthology.orchestra.run.smithi042.stdout:5944 read 0x83a3977 thru 0x83ae64c (0xacd6 bytes) 2024-09-18T00:59:44.402 INFO:teuthology.orchestra.run.smithi042.stdout:5945 writesame 0xa0ec3e8 thru 0xa0f38ff data_size 0xc(0x7518 bytes) 2024-09-18T00:59:44.410 INFO:teuthology.orchestra.run.smithi042.stdout:5946 read 0x245c4c9 thru 0x246ba54 (0xf58c bytes) 2024-09-18T00:59:44.410 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:44.413 INFO:teuthology.orchestra.run.smithi042.stdout:5949 trunc from 0xe7bccd3 to 0xd0d7f7c 2024-09-18T00:59:44.430 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:44.444 INFO:teuthology.orchestra.run.smithi042.stdout:5952 writesame 0xe6ad174 thru 0xe6b70f2 data_size 0x13(0x9f7f bytes) 2024-09-18T00:59:44.444 INFO:teuthology.orchestra.run.smithi042.stdout:5954 writesame 0x477d942 thru 0x4783f6e data_size 0x1(0x662d bytes) 2024-09-18T00:59:44.456 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:44.457+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:44.458 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:44.458 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:44.460 INFO:teuthology.orchestra.run.smithi042.stdout:5955 read 0x652ef18 thru 0x653ee3f (0xff28 bytes) 2024-09-18T00:59:44.460 INFO:teuthology.orchestra.run.smithi042.stdout:5956 write 0xd393b9 thru 0xd41037 (0x7c7f bytes) 2024-09-18T00:59:44.463 INFO:teuthology.orchestra.run.smithi042.stdout:5957 punch from 0x774065a to 0x7748e2b, (0x87d1 bytes) 2024-09-18T00:59:44.464 INFO:teuthology.orchestra.run.smithi042.stdout:5958 punch from 0x24ea8fc to 0x24f3bd0, (0x92d4 bytes) 2024-09-18T00:59:44.464 INFO:teuthology.orchestra.run.smithi042.stdout:5959 read 0x1ae99c6 thru 0x1aeeef9 (0x5534 bytes) 2024-09-18T00:59:44.464 INFO:teuthology.orchestra.run.smithi042.stdout:5960 read 0x1e8cfb2 thru 0x1e90510 (0x355f bytes) 2024-09-18T00:59:44.465 INFO:teuthology.orchestra.run.smithi042.stdout:5961 write 0x1b6008e thru 0x1b6d9e1 (0xd954 bytes) 2024-09-18T00:59:44.475 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:44.476 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:44.480 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:44.481 INFO:teuthology.orchestra.run.smithi042.stdout:5963 punch from 0xa7870cd to 0xa793516, (0xc449 bytes) 2024-09-18T00:59:44.481 INFO:teuthology.orchestra.run.smithi042.stdout:5965 write 0x2305916 thru 0x231351a (0xdc05 bytes) 2024-09-18T00:59:44.483 INFO:teuthology.orchestra.run.smithi042.stdout:5966 writesame 0xe085e09 thru 0xe08ff84 data_size 0x6a(0xa17c bytes) 2024-09-18T00:59:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:5968 writesame 0x17aff31 thru 0x17b5d9e data_size 0x66(0x5e6e bytes) 2024-09-18T00:59:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:5969 read 0x81cfcb2 thru 0x81d0894 (0xbe3 bytes) 2024-09-18T00:59:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:5970 punch from 0xd6a7e90 to 0xd6b077a, (0x88ea bytes) 2024-09-18T00:59:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:5972 read 0x4ce71a9 thru 0x4cf2d92 (0xbbea bytes) 2024-09-18T00:59:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:5973 write 0x3ad441f thru 0x3ae1e40 (0xda22 bytes) 2024-09-18T00:59:44.494 INFO:teuthology.orchestra.run.smithi042.stdout:5974 read 0x7d1e0a2 thru 0x7d29974 (0xb8d3 bytes) 2024-09-18T00:59:44.494 INFO:teuthology.orchestra.run.smithi042.stdout:5975 read 0xaec5864 thru 0xaecdc77 (0x8414 bytes) 2024-09-18T00:59:44.494 INFO:teuthology.orchestra.run.smithi042.stdout:5976 writesame 0x70237aa thru 0x702db56 data_size 0x1(0xa3ad bytes) 2024-09-18T00:59:44.552 INFO:teuthology.orchestra.run.smithi042.stdout:5977 write 0xb92a0 thru 0xc1e15 (0x8b76 bytes) 2024-09-18T00:59:44.557 INFO:teuthology.orchestra.run.smithi042.stdout:5979 read 0xd7237d3 thru 0xd732a4c (0xf27a bytes) 2024-09-18T00:59:44.558 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:44.557+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:44.558 INFO:teuthology.orchestra.run.smithi042.stdout:5980 read 0xd5a9295 thru 0xd5ac714 (0x3480 bytes) 2024-09-18T00:59:44.558 INFO:teuthology.orchestra.run.smithi042.stdout:5982 write 0xc2fd41f thru 0xc30bec6 (0xeaa8 bytes) 2024-09-18T00:59:44.567 INFO:teuthology.orchestra.run.smithi042.stdout:5983 read 0xd6a362 thru 0xd74efd (0xab9c bytes) 2024-09-18T00:59:44.574 INFO:teuthology.orchestra.run.smithi042.stdout:5986 trunc from 0xe6b70f3 to 0xa507822 2024-09-18T00:59:44.603 INFO:teuthology.orchestra.run.smithi042.stdout:5987 punch from 0x19923c4 to 0x19979d5, (0x5611 bytes) 2024-09-18T00:59:44.603 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-18T00:59:44.603 INFO:teuthology.orchestra.run.smithi042.stdout:5989 writesame 0x4731567 thru 0x4734ec9 data_size 0x3(0x3963 bytes) 2024-09-18T00:59:44.607 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:44.607 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:44.610 INFO:teuthology.orchestra.run.smithi042.stdout:5990 read 0x39b4676 thru 0x39b9e74 (0x57ff bytes) 2024-09-18T00:59:44.611 INFO:teuthology.orchestra.run.smithi042.stdout:5992 read 0x63a187e thru 0x63a7a9e (0x6221 bytes) 2024-09-18T00:59:44.622 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:44.622 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:44.626 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:44.629 INFO:teuthology.orchestra.run.smithi042.stdout:5993 write 0xda74e07 thru 0xda849d3 (0xfbcd bytes) 2024-09-18T00:59:44.639 INFO:teuthology.orchestra.run.smithi042.stdout:5994 read 0xac37d0f thru 0xac43c64 (0xbf56 bytes) 2024-09-18T00:59:44.639 INFO:teuthology.orchestra.run.smithi042.stdout:5996 writesame 0x29ce2cb thru 0x29d18ec data_size 0xd(0x3622 bytes) 2024-09-18T00:59:44.641 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:44.641 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:44.645 INFO:teuthology.orchestra.run.smithi042.stdout:5999 trunc from 0xda849d4 to 0xb50b8ff 2024-09-18T00:59:44.658 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:44.657+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:44.659 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:44.659 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:44.671 INFO:teuthology.orchestra.run.smithi042.stdout:6000 read 0x8bfaf9f thru 0x8c051ec (0xa24e bytes) 2024-09-18T00:59:44.676 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:44.677 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:44.697 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-18T00:59:44.699 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:44.700 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:44.721 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:44.721 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:44.742 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:44.743 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:44.743 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:44.743 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:44.743 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:44.743 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:44.759 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:44.757+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:44.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:44.764 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:44.860 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:44.857+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:44.882 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:44.882 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:44.898 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:44.960 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:44.961+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:45.061 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:45.061 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:45.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:45.061 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:45.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.061 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 141.76753071600001, 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:59:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:59:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:59:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:59:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.571168226, 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.571072899, 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:59:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:59:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:59:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:59:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.57104746, 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:59:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:59:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.571024937, 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.571001497, 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:59:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.570977687, 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:59:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:59:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.570955322, 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:59:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:59:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:59:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:59:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:59:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:59:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.570928705, 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:59:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:59:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.566756191, 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:59:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.566710626, 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:59:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.56668428499999, 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:59:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:59:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.566661615, 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.566643242, 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:59:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:59:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.56662345399999, 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:59:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.566596584, 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:59:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:59:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.490406114999999, 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.079341019000001, 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T00:59:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:140635 3.d 3:b08054e4:::rbd_object_map.117ced55205b.0000000000000023:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.718227625000001, 2024-09-18T00:59:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86222047999999996, 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 140635 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338625+0000", 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338634+0000", 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9099999999999994e-06 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338657+0000", 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3672999999999998e-05 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338675+0000", 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7589999999999999e-05 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338767+0000", 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1896000000000002e-05 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338776+0000", 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.38e-06 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200213+0000", 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86143668200000001 2024-09-18T00:59:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200432+0000", 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021884199999999999 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200844+0000", 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000411887 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:155664 3.d 3:b496760f:::rbd_data.117c1d78ece1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:40.905926+0000", 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.1509245210000003, 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046053647000000003, 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 155664 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905927+0000", 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.175e-06 2024-09-18T00:59:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909267+0000", 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003339266 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909287+0000", 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605e-05 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909297+0000", 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:59:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909363+0000", 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6437000000000001e-05 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909483+0000", 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119861 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909868+0000", 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038564099999999999 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.917828+0000", 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079592570000000008 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951788+0000", 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033960314999999998 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951895+0000", 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107316 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951980+0000", 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4460000000000001e-05 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.158 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:45.159 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:45.057+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:45.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:45.161 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:45.161+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:45.264 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:45.261+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:45.273 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:59:45.276 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-18T00:59:45.345 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:45.345 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:45.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:45.402 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:45.401+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:45.464 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:45.465+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:45.522 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:45.522 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:45.522 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:45.523 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:45.523 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:45.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:45.565 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:45.565+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:45.660 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.581160297, 2024-09-18T00:59:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:59:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:59:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.58023070599999, 2024-09-18T00:59:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:59:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:59:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:59:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.580216646, 2024-09-18T00:59:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:59:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:59:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:59:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:59:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:59:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:59:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:59:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:59:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:59:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.578926071, 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.578905599, 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.576111146, 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:59:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:59:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.576095962, 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:59:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.576073016, 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.571895377, 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.57187415999999, 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:59:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.571852777, 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:59:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:59:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:59:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.56795292300001, 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.56793872199999, 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.567920685, 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:59:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.567906619, 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:59:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:59:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:59:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:59:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:59:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.563881268, 2024-09-18T00:59:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:59:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:59:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:59:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:59:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:59:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:59:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.56386725599999, 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.563853292, 2024-09-18T00:59:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.801995338, 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.676518967000007, 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:59:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:59:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:59:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:59:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:45.755 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:45.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:45.757 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:45.753+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:45.858 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:45.857+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:45.926 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:45.939 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:45.986 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:45.985+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:46.058 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:46.057+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:46.062 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:46.062 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:46.062 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:46.063 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:46.063 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:46.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:46.159 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:46.157+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:46.250 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98052 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4179968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.806616755999997, 2024-09-18T00:59:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131360583, 2024-09-18T00:59:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98052 2024-09-18T00:59:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440037+0000", 2024-09-18T00:59:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440040+0000", 2024-09-18T00:59:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7159999999999999e-06 2024-09-18T00:59:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440048+0000", 2024-09-18T00:59:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7169999999999997e-06 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440056+0000", 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1860000000000003e-06 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.570989+0000", 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13093384999999999 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571175+0000", 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018504799999999999 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571397+0000", 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022245899999999999 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98053 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4466688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.806588886999997, 2024-09-18T00:59:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13182195299999999, 2024-09-18T00:59:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98053 2024-09-18T00:59:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440065+0000", 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440068+0000", 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6699999999999998e-06 2024-09-18T00:59:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440076+0000", 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7069999999999998e-06 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440083+0000", 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9929999999999998e-06 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571475+0000", 2024-09-18T00:59:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13139158100000001 2024-09-18T00:59:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571657+0000", 2024-09-18T00:59:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018247099999999999 2024-09-18T00:59:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571886+0000", 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022892400000000001 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98054 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2113536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.806561027000001, 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13228911600000001, 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98054 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440093+0000", 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440096+0000", 2024-09-18T00:59:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440105+0000", 2024-09-18T00:59:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7739999999999993e-06 2024-09-18T00:59:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440112+0000", 2024-09-18T00:59:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2130000000000002e-06 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571970+0000", 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13185812199999999 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572159+0000", 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000188665 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572381+0000", 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022273200000000001 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98055 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2400256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.806530846999998, 2024-09-18T00:59:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273966600000001, 2024-09-18T00:59:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98055 2024-09-18T00:59:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440123+0000", 2024-09-18T00:59:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440126+0000", 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440137+0000", 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0543000000000001e-05 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440144+0000", 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572457+0000", 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132312807 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572636+0000", 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882299999999999 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572862+0000", 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022645200000000001 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98056 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2686976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.806500270000001, 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13308801200000001, 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98056 2024-09-18T00:59:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440154+0000", 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5300000000000004e-07 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440156+0000", 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.903e-06 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440165+0000", 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7600000000000008e-06 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440174+0000", 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6969999999999999e-06 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572960+0000", 2024-09-18T00:59:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132785984 2024-09-18T00:59:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573087+0000", 2024-09-18T00:59:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012728699999999999 2024-09-18T00:59:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573241+0000", 2024-09-18T00:59:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153728 2024-09-18T00:59:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98057 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2973696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.806470677, 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13357376900000001, 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98057 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440183+0000", 2024-09-18T00:59:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-18T00:59:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440186+0000", 2024-09-18T00:59:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8700000000000001e-06 2024-09-18T00:59:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440195+0000", 2024-09-18T00:59:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7099999999999996e-06 2024-09-18T00:59:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440202+0000", 2024-09-18T00:59:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2429999999999999e-06 2024-09-18T00:59:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573338+0000", 2024-09-18T00:59:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13313578400000001 2024-09-18T00:59:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573526+0000", 2024-09-18T00:59:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018780399999999999 2024-09-18T00:59:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573756+0000", 2024-09-18T00:59:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230755 2024-09-18T00:59:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98058 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.806442793999999, 2024-09-18T00:59:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13387749400000001, 2024-09-18T00:59:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98058 2024-09-18T00:59:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440627+0000", 2024-09-18T00:59:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041660699999999999 2024-09-18T00:59:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440634+0000", 2024-09-18T00:59:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.596e-06 2024-09-18T00:59:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440650+0000", 2024-09-18T00:59:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6164e-05 2024-09-18T00:59:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440660+0000", 2024-09-18T00:59:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.023e-05 2024-09-18T00:59:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573833+0000", 2024-09-18T00:59:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13317317000000001 2024-09-18T00:59:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573945+0000", 2024-09-18T00:59:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111731 2024-09-18T00:59:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574088+0000", 2024-09-18T00:59:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000142996 2024-09-18T00:59:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98059 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3547136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.805975447000002, 2024-09-18T00:59:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13391484000000001, 2024-09-18T00:59:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98059 2024-09-18T00:59:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441760+0000", 2024-09-18T00:59:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001082185 2024-09-18T00:59:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441780+0000", 2024-09-18T00:59:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0463e-05 2024-09-18T00:59:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441829+0000", 2024-09-18T00:59:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8763000000000001e-05 2024-09-18T00:59:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441849+0000", 2024-09-18T00:59:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9423e-05 2024-09-18T00:59:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574173+0000", 2024-09-18T00:59:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13232460400000001 2024-09-18T00:59:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574367+0000", 2024-09-18T00:59:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019403000000000001 2024-09-18T00:59:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574593+0000", 2024-09-18T00:59:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000225372 2024-09-18T00:59:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98060 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3833856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.804773365999999, 2024-09-18T00:59:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13320725899999999, 2024-09-18T00:59:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98060 2024-09-18T00:59:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441881+0000", 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6160000000000001e-06 2024-09-18T00:59:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441889+0000", 2024-09-18T00:59:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8439999999999998e-06 2024-09-18T00:59:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441909+0000", 2024-09-18T00:59:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9856000000000001e-05 2024-09-18T00:59:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441925+0000", 2024-09-18T00:59:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6039999999999999e-05 2024-09-18T00:59:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574669+0000", 2024-09-18T00:59:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13274412799999999 2024-09-18T00:59:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574860+0000", 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019038599999999999 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575087+0000", 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022738899999999999 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98061 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4120576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.804706609999997, 2024-09-18T00:59:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13362664099999999, 2024-09-18T00:59:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98061 2024-09-18T00:59:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441948+0000", 2024-09-18T00:59:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:59:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441954+0000", 2024-09-18T00:59:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3099999999999997e-06 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441981+0000", 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7095999999999999e-05 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441998+0000", 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6237000000000001e-05 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575165+0000", 2024-09-18T00:59:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13316721200000001 2024-09-18T00:59:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575355+0000", 2024-09-18T00:59:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000190218 2024-09-18T00:59:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575573+0000", 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021824799999999999 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98062 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4407296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.804634641, 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13404179099999999, 2024-09-18T00:59:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98062 2024-09-18T00:59:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445789+0000", 2024-09-18T00:59:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037705569999999999 2024-09-18T00:59:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445805+0000", 2024-09-18T00:59:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.558e-05 2024-09-18T00:59:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445846+0000", 2024-09-18T00:59:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445871+0000", 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5072999999999999e-05 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575651+0000", 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12977999600000001 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575835+0000", 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000183931 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576060+0000", 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022558800000000001 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98064 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2115584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.800670201999999, 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13103894799999999, 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98064 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445985+0000", 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445991+0000", 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446014+0000", 2024-09-18T00:59:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3056000000000001e-05 2024-09-18T00:59:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446030+0000", 2024-09-18T00:59:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5662999999999999e-05 2024-09-18T00:59:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576622+0000", 2024-09-18T00:59:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13059273900000001 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576805+0000", 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018304899999999999 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577022+0000", 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216481 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98065 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2402304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.800603086000002, 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131457463, 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98065 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446051+0000", 2024-09-18T00:59:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1260000000000001e-06 2024-09-18T00:59:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446057+0000", 2024-09-18T00:59:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2040000000000002e-06 2024-09-18T00:59:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446077+0000", 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446092+0000", 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5124e-05 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577098+0000", 2024-09-18T00:59:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13100595100000001 2024-09-18T00:59:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577284+0000", 2024-09-18T00:59:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000186152 2024-09-18T00:59:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577508+0000", 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022314000000000001 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98066 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2689024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.80054088, 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13189522100000001, 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98066 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446113+0000", 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0240000000000001e-06 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446120+0000", 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446137+0000", 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7479e-05 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446152+0000", 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4633e-05 2024-09-18T00:59:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577587+0000", 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13143572100000001 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577786+0000", 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019872999999999999 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578008+0000", 2024-09-18T00:59:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022141399999999999 2024-09-18T00:59:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98067 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2975744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.800483217, 2024-09-18T00:59:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132319567, 2024-09-18T00:59:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98067 2024-09-18T00:59:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446171+0000", 2024-09-18T00:59:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446178+0000", 2024-09-18T00:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699999999999997e-06 2024-09-18T00:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446196+0000", 2024-09-18T00:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7989999999999999e-05 2024-09-18T00:59:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446213+0000", 2024-09-18T00:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7493999999999999e-05 2024-09-18T00:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578083+0000", 2024-09-18T00:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13186995000000001 2024-09-18T00:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578266+0000", 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018321099999999999 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578490+0000", 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022316600000000001 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98068 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3262464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.800422310999998, 2024-09-18T00:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273661, 2024-09-18T00:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98068 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446232+0000", 2024-09-18T00:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.187e-06 2024-09-18T00:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446238+0000", 2024-09-18T00:59:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446257+0000", 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446272+0000", 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5532999999999998e-05 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578569+0000", 2024-09-18T00:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132296631 2024-09-18T00:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578749+0000", 2024-09-18T00:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017978300000000001 2024-09-18T00:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578967+0000", 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218736 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98069 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3549184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.800362638000003, 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13311771, 2024-09-18T00:59:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98069 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446292+0000", 2024-09-18T00:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446298+0000", 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4470000000000003e-06 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446340+0000", 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1705999999999997e-05 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446358+0000", 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7873999999999999e-05 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579072+0000", 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13271396399999999 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579213+0000", 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014133900000000001 2024-09-18T00:59:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579408+0000", 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019492700000000001 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98070 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3835904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.800269194999998, 2024-09-18T00:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133517145, 2024-09-18T00:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98070 2024-09-18T00:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446385+0000", 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446393+0000", 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2629999999999997e-06 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446413+0000", 2024-09-18T00:59:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9930000000000001e-05 2024-09-18T00:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446428+0000", 2024-09-18T00:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5336e-05 2024-09-18T00:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579494+0000", 2024-09-18T00:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13306637399999999 2024-09-18T00:59:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579682+0000", 2024-09-18T00:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018740900000000001 2024-09-18T00:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579901+0000", 2024-09-18T00:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021946300000000001 2024-09-18T00:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98071 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.800206029000002, 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133931615, 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98071 2024-09-18T00:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449904+0000", 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003456822 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449945+0000", 2024-09-18T00:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0896000000000003e-05 2024-09-18T00:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449971+0000", 2024-09-18T00:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5996999999999999e-05 2024-09-18T00:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449986+0000", 2024-09-18T00:59:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.49e-05 2024-09-18T00:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579978+0000", 2024-09-18T00:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.129992723 2024-09-18T00:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580157+0000", 2024-09-18T00:59:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179006 2024-09-18T00:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580379+0000", 2024-09-18T00:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022127099999999999 2024-09-18T00:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98073 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4696064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.796596901999997, 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13113503100000001, 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98073 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450057+0000", 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450060+0000", 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5329999999999999e-06 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450072+0000", 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1919999999999999e-05 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450080+0000", 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9500000000000001e-06 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580951+0000", 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.130871128 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581057+0000", 2024-09-18T00:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010501399999999999 2024-09-18T00:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581191+0000", 2024-09-18T00:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013479900000000001 2024-09-18T00:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:46.366 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:46.367 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:46.368 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:46.365+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:46.468 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:46.465+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:46.510 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:46.511 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:46.511 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:46.511 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:46.525 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:46.572 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:46.573+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:46.645 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:46.645 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:46.645 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:46.645 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:46.645 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:46.659 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:46.700 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:46.701+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:46.770 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:46.769+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:46.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:46.825 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:46.826 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:46.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:46.826 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:46.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.826 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:46.826 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:59:46.826 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 118.04831591600001, 2024-09-18T00:59:46.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:59:46.826 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.826 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.827 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:46.827 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.827 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.827 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.827 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:59:46.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.827 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:59:46.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:59:46.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:59:46.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:59:46.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:59:46.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:59:46.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:59:46.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:59:46.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:59:46.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:59:46.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:59:46.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:59:46.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:59:46.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:59:46.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:59:46.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:59:46.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:59:46.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:46.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:59:46.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:59:46.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:46.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:59:46.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:59:46.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:46.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:59:46.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:59:46.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:59:46.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:59:46.836 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.836 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.836 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.836 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87406 3.19 3:9d734bda:::rbd_data.117c8934add3.000000000000007c:head [stat,delete] snapc 1f=[1f] RETRY=8 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:59:46.837 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:59:46.837 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 58.400392793999998, 2024-09-18T00:59:46.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097024104999999999, 2024-09-18T00:59:46.837 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.837 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.837 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:46.837 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.837 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.838 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.838 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87406 2024-09-18T00:59:46.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.838 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:59:46.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:59:46.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425532+0000", 2024-09-18T00:59:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027458309999999998 2024-09-18T00:59:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425537+0000", 2024-09-18T00:59:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2839999999999997e-06 2024-09-18T00:59:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425551+0000", 2024-09-18T00:59:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3682999999999999e-05 2024-09-18T00:59:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425559+0000", 2024-09-18T00:59:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5329999999999997e-06 2024-09-18T00:59:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515035+0000", 2024-09-18T00:59:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.089476411000000006 2024-09-18T00:59:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515168+0000", 2024-09-18T00:59:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013309699999999999 2024-09-18T00:59:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:59:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515475+0000", 2024-09-18T00:59:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030662199999999999 2024-09-18T00:59:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519553+0000", 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040781259999999996 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519665+0000", 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112669 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519748+0000", 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2270999999999999e-05 2024-09-18T00:59:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519810+0000", 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2577999999999996e-05 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87414 3.19 3:9c145b25:::rbd_data.117c8934add3.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 58.397569255000001, 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095886693999999995, 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.846 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87414 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425618+0000", 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9810000000000003e-06 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425625+0000", 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0870000000000001e-06 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425630+0000", 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.262e-06 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518417+0000", 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092787816999999995 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518593+0000", 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017569100000000001 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518939+0000", 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034572299999999998 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521320+0000", 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023809619999999999 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521343+0000", 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3317000000000001e-05 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521422+0000", 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480999999999996e-05 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521496+0000", 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4909999999999999e-05 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99159 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 393216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369266+0000", 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.453912905000003, 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096008205999999999, 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99159 2024-09-18T00:59:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369409+0000", 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014342599999999999 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369415+0000", 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.31e-06 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369426+0000", 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1677e-05 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369435+0000", 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.464949+0000", 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095513718999999997 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465099+0000", 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014984300000000001 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465274+0000", 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017554499999999999 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99160 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2336768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369451+0000", 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.453728335999998, 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096064584999999994, 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99160 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369564+0000", 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113309 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369569+0000", 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5269999999999998e-06 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369582+0000", 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2952999999999999e-05 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369592+0000", 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0939999999999997e-06 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465328+0000", 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095736285000000004 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465403+0000", 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5498000000000001e-05 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465515+0000", 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111919 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99161 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 413696~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369608+0000", 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.453570706999997, 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096302270999999995, 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99161 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373792+0000", 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004184063 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373801+0000", 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3729999999999993e-06 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373818+0000", 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7280000000000001e-05 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373829+0000", 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0540000000000001e-05 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465596+0000", 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.091767910999999994 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465740+0000", 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143683 2024-09-18T00:59:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465911+0000", 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000170421 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99172 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2459648~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.448975363999999, 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095842035000000006, 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99172 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9299999999999997e-07 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374207+0000", 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8899999999999999e-06 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374216+0000", 2024-09-18T00:59:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3669999999999997e-06 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374223+0000", 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9759999999999998e-06 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469763+0000", 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095540307000000005 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469894+0000", 2024-09-18T00:59:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013135600000000001 2024-09-18T00:59:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470046+0000", 2024-09-18T00:59:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015144600000000001 2024-09-18T00:59:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99173 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2480128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374231+0000", 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.448947803999999, 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096195265000000002, 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99173 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374232+0000", 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4600000000000004e-07 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374235+0000", 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8569999999999999e-06 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374243+0000", 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5930000000000006e-06 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374250+0000", 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9330000000000004e-06 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470113+0000", 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095862952000000001 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470259+0000", 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014534300000000001 2024-09-18T00:59:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470426+0000", 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016784100000000001 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99174 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2500608~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374259+0000", 2024-09-18T00:59:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.448919498000002, 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096540567999999993, 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99174 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374260+0000", 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2600000000000002e-07 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374263+0000", 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8770000000000002e-06 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374272+0000", 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6070000000000008e-06 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374279+0000", 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8430000000000004e-06 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470492+0000", 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096213513000000001 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470632+0000", 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013954899999999999 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470800+0000", 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016845299999999999 2024-09-18T00:59:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99175 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 536576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374287+0000", 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.448891602000003, 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096867154999999996, 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99175 2024-09-18T00:59:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374288+0000", 2024-09-18T00:59:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9999999999999997e-07 2024-09-18T00:59:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374291+0000", 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.92e-06 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374300+0000", 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5939999999999994e-06 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374307+0000", 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9829999999999999e-06 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470858+0000", 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096551085999999994 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470990+0000", 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013223900000000001 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471155+0000", 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016463299999999999 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99176 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2521088~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374315+0000", 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.448863652, 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097204272999999994, 2024-09-18T00:59:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99176 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374316+0000", 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1999999999999999e-07 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374336+0000", 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9879999999999999e-05 2024-09-18T00:59:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374346+0000", 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0353999999999999e-05 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374354+0000", 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7130000000000005e-06 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471210+0000", 2024-09-18T00:59:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096856271999999993 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471350+0000", 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013940900000000001 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471520+0000", 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000169925 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99177 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 557056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374364+0000", 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.448814788999996, 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097507736999999997, 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99177 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377919+0000", 2024-09-18T00:59:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035553070000000002 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377927+0000", 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4569999999999999e-06 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377943+0000", 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5563e-05 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377953+0000", 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0413e-05 2024-09-18T00:59:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471578+0000", 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093625036999999994 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471710+0000", 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013170900000000001 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471872+0000", 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162251 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99183 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 618496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378152+0000", 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.445026753, 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095979540000000002, 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99183 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378153+0000", 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.23e-07 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378156+0000", 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9340000000000002e-06 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378164+0000", 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4630000000000001e-06 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378173+0000", 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8130000000000002e-06 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473851+0000", 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678257000000003 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473982+0000", 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000130875 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474132+0000", 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014947499999999999 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99184 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2603008~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378209+0000", 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.444970210999998, 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096289871999999999, 2024-09-18T00:59:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99184 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378210+0000", 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2999999999999999e-07 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378213+0000", 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378223+0000", 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378230+0000", 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5830000000000001e-06 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474192+0000", 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961742000000003 2024-09-18T00:59:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474324+0000", 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000132099 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474499+0000", 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174551 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99185 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 638976~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378243+0000", 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.444935784000002, 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096618187999999994, 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99185 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378244+0000", 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5700000000000001e-07 2024-09-18T00:59:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378248+0000", 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5130000000000001e-06 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378256+0000", 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8429999999999999e-06 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378263+0000", 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9870000000000002e-06 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474554+0000", 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096290693999999996 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474692+0000", 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013761900000000001 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474861+0000", 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016967499999999999 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99186 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 659456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.444906234000001, 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096958847000000001, 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99186 2024-09-18T00:59:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1699999999999997e-07 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378277+0000", 2024-09-18T00:59:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0800000000000002e-06 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378285+0000", 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5699999999999993e-06 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378292+0000", 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9600000000000003e-06 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474924+0000", 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096632207999999997 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475067+0000", 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143175 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475232+0000", 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016413699999999999 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99187 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 679936~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378301+0000", 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.444877961000003, 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097293710000000005, 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99187 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T00:59:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378302+0000", 2024-09-18T00:59:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T00:59:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378305+0000", 2024-09-18T00:59:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.066e-06 2024-09-18T00:59:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378315+0000", 2024-09-18T00:59:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0443e-05 2024-09-18T00:59:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378323+0000", 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3239999999999997e-06 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475291+0000", 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096968726000000005 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475430+0000", 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013847499999999999 2024-09-18T00:59:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475595+0000", 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016497899999999999 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99188 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 700416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378332+0000", 2024-09-18T00:59:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.444847125000003, 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097626477000000003, 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99188 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381945+0000", 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036127569999999999 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381960+0000", 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.554e-05 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381977+0000", 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6733000000000001e-05 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381987+0000", 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.031e-05 2024-09-18T00:59:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475651+0000", 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093664175000000002 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475789+0000", 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000137181 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475958+0000", 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016978100000000001 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99195 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 761856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.440974320000002, 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961833999999996, 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99195 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382208+0000", 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0469999999999998e-06 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382217+0000", 2024-09-18T00:59:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5159999999999994e-06 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382224+0000", 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.64e-06 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.477889+0000", 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095664955999999995 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478012+0000", 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012299899999999999 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478166+0000", 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015497899999999999 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99196 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 88064~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382233+0000", 2024-09-18T00:59:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.440946390000001, 2024-09-18T00:59:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096312801000000003, 2024-09-18T00:59:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99196 2024-09-18T00:59:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T00:59:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T00:59:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385636+0000", 2024-09-18T00:59:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034037450000000001 2024-09-18T00:59:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385642+0000", 2024-09-18T00:59:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9900000000000002e-06 2024-09-18T00:59:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385659+0000", 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6753e-05 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385669+0000", 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9170000000000001e-06 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478225+0000", 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092555514000000005 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478366+0000", 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014198699999999999 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478545+0000", 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017889499999999999 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:46.931 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:46.932 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:46.933 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:46.933+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:46.943 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:46.944 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:46.960 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:46.960 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:46.977 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:46.977 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:46.994 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:46.995 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:47.016 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:47.016 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:47.032 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:47.029+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:47.041 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:47.041 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:47.061 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:47.062 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:47.082 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:47.083 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:47.107 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:47.108 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:47.122 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:47.142 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:47.141+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:47.234 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:47.233+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:47.244 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:47.244 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:47.258 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:47.302 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:59:47.350 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:47.349+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:47.393 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:47.393 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:47.407 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:47.436 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:47.433+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:47.536 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:47.536 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:47.537 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:47.537+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:47.551 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:47.637 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:47.637+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:47.716 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:47.716 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:47.730 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:47.776 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:47.777+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:47.839 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:47.837+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:47.902 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:47.903 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:47.917 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:47.940 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:47.937+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:48.041 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:48.041+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:48.081 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:48.081 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:48.095 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:48.141 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:48.141+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:48.242 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:48.241+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:48.260 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:48.260 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:48.276 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:48.287 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:59:48.362 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:48.361+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:48.443 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:48.441+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:48.454 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:48.454 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:48.469 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:48.544 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:48.541+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:48.605 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:48.606 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:48.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:48.645 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:48.645+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:48.747 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:48.745+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:48.789 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:48.789 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:48.804 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:48.847 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:48.845+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:48.949 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:48.949+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:48.974 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:48.974 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:48.990 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:49.050 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:49.049+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:49.101 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:49.133 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:49.151 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:49.149+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:49.242 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:49.252 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:49.253+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:49.259 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:49.259 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:49.276 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:49.292 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:49.293 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:49.301 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:49.301 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:49.301 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:49.301 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:49.302 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:49.323 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:49.325 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:49.326 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:49.349 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:49.353 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:49.353+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:49.370 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:49.370 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:49.392 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:49.392 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:49.443 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:49.443 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 146.15208471400001, 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.444 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:59:49.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:59:49.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:49.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:59:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:59:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:59:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:59:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:59:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:59:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:49.451 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.955722224, 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:59:49.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.455 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.955626897, 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.456 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.955601458, 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:59:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.95557893500001, 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:59:49.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:59:49.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.955555495, 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.469 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:59:49.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.473 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.955531685, 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:59:49.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:59:49.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:59:49.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.95550932, 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.478 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:59:49.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:59:49.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.955482703, 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:59:49.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.951310189, 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.487 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:59:49.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:59:49.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:59:49.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.951264624, 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:59:49.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:59:49.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.951238283, 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.499 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.951215613, 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:59:49.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:59:49.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.95119724, 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.504 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:59:49.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.951177452, 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.509 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:59:49.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:49.513 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.951150582, 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:59:49.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:59:49.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:59:49.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:59:49.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:59:49.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:59:49.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:59:49.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.518 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.519 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T00:59:49.519 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:49.519 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.874960113, 2024-09-18T00:59:49.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T00:59:49.519 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T00:59:49.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.463895016999999, 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.526 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T00:59:49.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T00:59:49.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:49.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:140635 3.d 3:b08054e4:::rbd_object_map.117ced55205b.0000000000000023:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-18T00:59:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.102781622999998, 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86222047999999996, 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 140635 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:49.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:49.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338625+0000", 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338634+0000", 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9099999999999994e-06 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338657+0000", 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3672999999999998e-05 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338675+0000", 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7589999999999999e-05 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338767+0000", 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1896000000000002e-05 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338776+0000", 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.38e-06 2024-09-18T00:59:49.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200213+0000", 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86143668200000001 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200432+0000", 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021884199999999999 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200844+0000", 2024-09-18T00:59:49.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000411887 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:155664 3.d 3:b496760f:::rbd_data.117c1d78ece1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:40.905926+0000", 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.5354785189999998, 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046053647000000003, 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:49.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 155664 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T00:59:49.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905927+0000", 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.175e-06 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909267+0000", 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003339266 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:49.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909287+0000", 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605e-05 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909297+0000", 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909363+0000", 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6437000000000001e-05 2024-09-18T00:59:49.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909483+0000", 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119861 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909868+0000", 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038564099999999999 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.917828+0000", 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079592570000000008 2024-09-18T00:59:49.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951788+0000", 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033960314999999998 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951895+0000", 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107316 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:49.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951980+0000", 2024-09-18T00:59:49.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4460000000000001e-05 2024-09-18T00:59:49.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:49.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:49.544 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:49.546 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:49.546 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:49.545+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:49.646 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:49.645+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:49.726 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:49.726 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:49.726 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:49.726 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:49.741 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:49.748 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:49.745+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:49.849 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:49.845+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:49.874 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:49.875 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:49.889 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:49.949 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:49.949+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:50.050 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:50.049+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.986469989, 2024-09-18T00:59:50.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:50.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:59:50.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.985540398, 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:59:50.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:59:50.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:50.079 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.985526338, 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:50.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:59:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.984235763, 2024-09-18T00:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:59:50.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.984215291, 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:59:50.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:59:50.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:59:50.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.093 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.98142083800001, 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:59:50.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.981405654, 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:59:50.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.981382708, 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:59:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:59:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.97720506900001, 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:50.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:59:50.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.977183852, 2024-09-18T00:59:50.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:59:50.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.97716246900001, 2024-09-18T00:59:50.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:50.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:59:50.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.973262615, 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.973248414, 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:50.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:59:50.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.97323037699999, 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:50.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:59:50.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:50.135 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.973216311, 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:59:50.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.96919096000001, 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:59:50.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.144 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.969176948, 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:59:50.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.96916298399999, 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.149 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:59:50.153 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:50.154 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.20730503, 2024-09-18T00:59:50.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:59:50.154 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:50.154 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:50.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:59:50.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.081828659, 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:50.158 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:59:50.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:59:50.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:59:50.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:50.166 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:50.166 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:50.161+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:50.267 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:50.265+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:50.302 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6] out_osds: [2, 3, 0, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:59:50.302 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:59:50.302 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T00:59:50.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T00:59:50.322 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:59:50.355 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:50.356 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:50.356 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:50.367 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:50.365+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:50.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:50.468 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:50.469+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:50.554 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:50.569 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:50.569+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:50.570 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 132, 2024-09-18T00:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "82a96d76-ba83-49e2-a985-84604bd599ba", 2024-09-18T00:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-18T00:57:07.138732+0000", 2024-09-18T00:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-18T00:59:50.307802+0000", 2024-09-18T00:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-18T00:57:20.258605+0000", 2024-09-18T00:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-18T00:59:27.145670+0000", 2024-09-18T00:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T00:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5801984, 2024-09-18T00:59:50.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "noscrub", 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-18T00:59:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "jewel", 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:21.962521+0000", 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:50.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-18T00:59:50.641 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "19", 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:50.642 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:50.644 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:50.645 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 8, 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 8, 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.25, 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-18T00:59:50.646 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:25.721733+0000", 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-18T00:59:50.647 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "23", 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:50.648 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 23, 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:50.649 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:50.650 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:50.651 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 3, 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 3, 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.5, 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.5, 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.5, 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-18T00:59:50.652 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:35.599413+0000", 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:59:50.653 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "132", 2024-09-18T00:59:50.654 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 45, 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 132, 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T00:59:50.655 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.656 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T00:59:50.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 3, 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 3, 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.5, 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.5, 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.5, 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T00:59:50.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "3f359ce5-6f6e-4161-8554-8ac028347274", 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 95, 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.660 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.662 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/199763443", 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/199763443", 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/199763443", 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/199763443", 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:50.663 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "48687611-1556-4b6d-86af-7872494a91d7", 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 118, 2024-09-18T00:59:50.664 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/345149800", 2024-09-18T00:59:50.668 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/345149800", 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/345149800", 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/345149800", 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "9e6b1f90-ce9b-46d4-92fe-d457f7e415d5", 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:50.669 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/431239931", 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/431239931", 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/431239931", 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/431239931", 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5", 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 89, 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.676 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:50.677 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:50.678 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.679 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/1775474180", 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/1775474180", 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/1775474180", 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/1775474180", 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "802ccbeb-7351-4b38-832a-695117c7e493", 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:50.680 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 117, 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6808", 2024-09-18T00:59:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6809", 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6810", 2024-09-18T00:59:50.682 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6811", 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6814", 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:50.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6815", 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6812", 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:50.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6813", 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6809/4198401207", 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6811/4198401207", 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6815/4198401207", 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6813/4198401207", 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:50.685 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d958b3da-efa1-4d86-b81d-e78874441437", 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 117, 2024-09-18T00:59:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6824", 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6825", 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:50.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6826", 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6827", 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.688 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6830", 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6831", 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.689 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6828", 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6829", 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6825/2599825823", 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6827/2599825823", 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6831/2599825823", 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6829/2599825823", 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "51b3cea8-c686-4fd3-b3f4-ba936cf26b64", 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 117, 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6800", 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6801", 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.693 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6802", 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6803", 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6806", 2024-09-18T00:59:50.694 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6807", 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6804", 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:50.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6805", 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6801/3550656451", 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6803/3550656451", 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6807/3550656451", 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6805/3550656451", 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:50.696 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b02ead1c-837a-4b07-9ad1-4e903ed5bc4e", 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 106, 2024-09-18T00:59:50.697 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6816", 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6817", 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.698 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6818", 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6819", 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6822", 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6823", 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6820", 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6821", 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6817/2993352359", 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6819/2993352359", 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6823/2993352359", 2024-09-18T00:59:50.701 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6821/2993352359", 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:50.702 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.896477+0000", 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.941756+0000", 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T00:59:50.703 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.560180+0000", 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.077394+0000", 2024-09-18T00:59:50.704 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.996264+0000", 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T00:59:50.705 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.511686+0000", 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:50.706 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.411051+0000", 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.888092+0000", 2024-09-18T00:59:50.707 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-18T00:59:50.708 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [ 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 41, 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 5 2024-09-18T00:59:50.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [ 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 41, 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.710 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T00:59:50.711 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-18T00:59:50.711 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-18T00:59:50.711 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-18T00:59:50.711 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-18T00:59:50.711 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-18T00:59:50.711 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-18T00:59:50.711 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-18T00:59:50.711 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-18T00:59:50.711 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-18T00:59:50.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.711 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:50.714 INFO:tasks.thrashosds.thrasher:j is {'epoch': 132, 'fsid': '82a96d76-ba83-49e2-a985-84604bd599ba', 'created': '2024-09-18T00:57:07.138732+0000', 'modified': '2024-09-18T00:59:50.307802+0000', 'last_up_change': '2024-09-18T00:57:20.258605+0000', 'last_in_change': '2024-09-18T00:59:27.145670+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T00:57:21.962521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T00:57:25.721733+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.5, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T00:57:35.599413+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 132, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.5, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3f359ce5-6f6e-4161-8554-8ac028347274', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 199763443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 199763443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 199763443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 199763443}]}, 'public_addr': '172.21.15.42:6801/199763443', 'cluster_addr': '172.21.15.42:6803/199763443', 'heartbeat_back_addr': '172.21.15.42:6807/199763443', 'heartbeat_front_addr': '172.21.15.42:6805/199763443', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '48687611-1556-4b6d-86af-7872494a91d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 345149800}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 345149800}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 345149800}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 345149800}]}, 'public_addr': '172.21.15.42:6809/345149800', 'cluster_addr': '172.21.15.42:6811/345149800', 'heartbeat_back_addr': '172.21.15.42:6815/345149800', 'heartbeat_front_addr': '172.21.15.42:6813/345149800', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9e6b1f90-ce9b-46d4-92fe-d457f7e415d5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 431239931}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 431239931}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 431239931}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 431239931}]}, 'public_addr': '172.21.15.42:6825/431239931', 'cluster_addr': '172.21.15.42:6827/431239931', 'heartbeat_back_addr': '172.21.15.42:6831/431239931', 'heartbeat_front_addr': '172.21.15.42:6829/431239931', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 89, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 1775474180}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 1775474180}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 1775474180}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 1775474180}]}, 'public_addr': '172.21.15.42:6817/1775474180', 'cluster_addr': '172.21.15.42:6819/1775474180', 'heartbeat_back_addr': '172.21.15.42:6823/1775474180', 'heartbeat_front_addr': '172.21.15.42:6821/1775474180', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '802ccbeb-7351-4b38-832a-695117c7e493', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6809', 'nonce': 4198401207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6811', 'nonce': 4198401207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6815', 'nonce': 4198401207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6813', 'nonce': 4198401207}]}, 'public_addr': '172.21.15.195:6809/4198401207', 'cluster_addr': '172.21.15.195:6811/4198401207', 'heartbeat_back_addr': '172.21.15.195:6815/4198401207', 'heartbeat_front_addr': '172.21.15.195:6813/4198401207', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd958b3da-efa1-4d86-b81d-e78874441437', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6824', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6825', 'nonce': 2599825823}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6826', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6827', 'nonce': 2599825823}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6830', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6831', 'nonce': 2599825823}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6828', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6829', 'nonce': 2599825823}]}, 'public_addr': '172.21.15.195:6825/2599825823', 'cluster_addr': '172.21.15.195:6827/2599825823', 'heartbeat_back_addr': '172.21.15.195:6831/2599825823', 'heartbeat_front_addr': '172.21.15.195:6829/2599825823', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '51b3cea8-c686-4fd3-b3f4-ba936cf26b64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6800', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6801', 'nonce': 3550656451}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6803', 'nonce': 3550656451}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6807', 'nonce': 3550656451}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6805', 'nonce': 3550656451}]}, 'public_addr': '172.21.15.195:6801/3550656451', 'cluster_addr': '172.21.15.195:6803/3550656451', 'heartbeat_back_addr': '172.21.15.195:6807/3550656451', 'heartbeat_front_addr': '172.21.15.195:6805/3550656451', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b02ead1c-837a-4b07-9ad1-4e903ed5bc4e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6817', 'nonce': 2993352359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6818', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6819', 'nonce': 2993352359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6822', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6823', 'nonce': 2993352359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6820', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6821', 'nonce': 2993352359}]}, 'public_addr': '172.21.15.195:6817/2993352359', 'cluster_addr': '172.21.15.195:6819/2993352359', 'heartbeat_back_addr': '172.21.15.195:6823/2993352359', 'heartbeat_front_addr': '172.21.15.195:6821/2993352359', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.896477+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.941756+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.560180+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.077394+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.996264+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.511686+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.411051+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.888092+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 41, 'length': 5}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 41, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T00:59:50.714 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-18T00:59:50.715 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:50.715 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:50.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:50.715 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:50.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.715 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98052 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4179968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.715 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:50.715 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.253950566, 2024-09-18T00:59:50.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131360583, 2024-09-18T00:59:50.715 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98052 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440037+0000", 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440040+0000", 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7159999999999999e-06 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440048+0000", 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7169999999999997e-06 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440056+0000", 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1860000000000003e-06 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.570989+0000", 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13093384999999999 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571175+0000", 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018504799999999999 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571397+0000", 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022245899999999999 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98053 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4466688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.253922697, 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13182195299999999, 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98053 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440065+0000", 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440068+0000", 2024-09-18T00:59:50.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6699999999999998e-06 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440076+0000", 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7069999999999998e-06 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440083+0000", 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9929999999999998e-06 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571475+0000", 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13139158100000001 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571657+0000", 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018247099999999999 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571886+0000", 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022892400000000001 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98054 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2113536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.253894836999997, 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13228911600000001, 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98054 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440093+0000", 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440096+0000", 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:50.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440105+0000", 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7739999999999993e-06 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440112+0000", 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2130000000000002e-06 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571970+0000", 2024-09-18T00:59:50.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13185812199999999 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572159+0000", 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000188665 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572381+0000", 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022273200000000001 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98055 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2400256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.253864657000001, 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273966600000001, 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98055 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440123+0000", 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440126+0000", 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440137+0000", 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0543000000000001e-05 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440144+0000", 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-18T00:59:50.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572457+0000", 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132312807 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572636+0000", 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882299999999999 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572862+0000", 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022645200000000001 2024-09-18T00:59:50.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98056 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2686976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.253834079999997, 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13308801200000001, 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98056 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440154+0000", 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5300000000000004e-07 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440156+0000", 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.903e-06 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440165+0000", 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7600000000000008e-06 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440174+0000", 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6969999999999999e-06 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572960+0000", 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132785984 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573087+0000", 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012728699999999999 2024-09-18T00:59:50.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573241+0000", 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153728 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98057 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2973696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.253804486999996, 2024-09-18T00:59:50.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13357376900000001, 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98057 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440183+0000", 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440186+0000", 2024-09-18T00:59:50.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8700000000000001e-06 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440195+0000", 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7099999999999996e-06 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440202+0000", 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2429999999999999e-06 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573338+0000", 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13313578400000001 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573526+0000", 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018780399999999999 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573756+0000", 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230755 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.742 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98058 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.253776604000002, 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13387749400000001, 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.743 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98058 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440627+0000", 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041660699999999999 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440634+0000", 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.596e-06 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440650+0000", 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6164e-05 2024-09-18T00:59:50.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440660+0000", 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.023e-05 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573833+0000", 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13317317000000001 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573945+0000", 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111731 2024-09-18T00:59:50.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574088+0000", 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000142996 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98059 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3547136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.253309256999998, 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13391484000000001, 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98059 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441760+0000", 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001082185 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441780+0000", 2024-09-18T00:59:50.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0463e-05 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441829+0000", 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8763000000000001e-05 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441849+0000", 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9423e-05 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574173+0000", 2024-09-18T00:59:50.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13232460400000001 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574367+0000", 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019403000000000001 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574593+0000", 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000225372 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98060 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3833856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.252107176000003, 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13320725899999999, 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98060 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.752 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441881+0000", 2024-09-18T00:59:50.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6160000000000001e-06 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441889+0000", 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8439999999999998e-06 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441909+0000", 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9856000000000001e-05 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441925+0000", 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6039999999999999e-05 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574669+0000", 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13274412799999999 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574860+0000", 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019038599999999999 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575087+0000", 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022738899999999999 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98061 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4120576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.25204042, 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13362664099999999, 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98061 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441948+0000", 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441954+0000", 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3099999999999997e-06 2024-09-18T00:59:50.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441981+0000", 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7095999999999999e-05 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441998+0000", 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6237000000000001e-05 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575165+0000", 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13316721200000001 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575355+0000", 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000190218 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575573+0000", 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021824799999999999 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98062 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4407296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.760 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.251968451000003, 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13404179099999999, 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98062 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445789+0000", 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037705569999999999 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445805+0000", 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.558e-05 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445846+0000", 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445871+0000", 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5072999999999999e-05 2024-09-18T00:59:50.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575651+0000", 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12977999600000001 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575835+0000", 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000183931 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576060+0000", 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022558800000000001 2024-09-18T00:59:50.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98064 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2115584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.248004012000003, 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13103894799999999, 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98064 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445985+0000", 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445991+0000", 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446014+0000", 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3056000000000001e-05 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446030+0000", 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5662999999999999e-05 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576622+0000", 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13059273900000001 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576805+0000", 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018304899999999999 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577022+0000", 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216481 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98065 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2402304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.247936895999999, 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131457463, 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98065 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446051+0000", 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1260000000000001e-06 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446057+0000", 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2040000000000002e-06 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446077+0000", 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446092+0000", 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5124e-05 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577098+0000", 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13100595100000001 2024-09-18T00:59:50.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577284+0000", 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000186152 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577508+0000", 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022314000000000001 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98066 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2689024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.247874690000003, 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13189522100000001, 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98066 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.774 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446113+0000", 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0240000000000001e-06 2024-09-18T00:59:50.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446120+0000", 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446137+0000", 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7479e-05 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446152+0000", 2024-09-18T00:59:50.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4633e-05 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577587+0000", 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13143572100000001 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577786+0000", 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019872999999999999 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578008+0000", 2024-09-18T00:59:50.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022141399999999999 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98067 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2975744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.247817027000004, 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132319567, 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98067 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446171+0000", 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446178+0000", 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699999999999997e-06 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446196+0000", 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7989999999999999e-05 2024-09-18T00:59:50.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446213+0000", 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7493999999999999e-05 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578083+0000", 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13186995000000001 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578266+0000", 2024-09-18T00:59:50.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018321099999999999 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578490+0000", 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022316600000000001 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98068 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3262464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.247756121000002, 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273661, 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98068 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446232+0000", 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.187e-06 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446238+0000", 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446257+0000", 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446272+0000", 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5532999999999998e-05 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578569+0000", 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132296631 2024-09-18T00:59:50.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578749+0000", 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017978300000000001 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578967+0000", 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218736 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98069 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3549184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.247696447999999, 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13311771, 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98069 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446292+0000", 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446298+0000", 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4470000000000003e-06 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446340+0000", 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1705999999999997e-05 2024-09-18T00:59:50.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446358+0000", 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7873999999999999e-05 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579072+0000", 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13271396399999999 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579213+0000", 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014133900000000001 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579408+0000", 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019492700000000001 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98070 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3835904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.791 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.247603005000002, 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133517145, 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98070 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446385+0000", 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-09-18T00:59:50.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446393+0000", 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2629999999999997e-06 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446413+0000", 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9930000000000001e-05 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446428+0000", 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5336e-05 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579494+0000", 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13306637399999999 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579682+0000", 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018740900000000001 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579901+0000", 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021946300000000001 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98071 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.247539838999998, 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133931615, 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98071 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449904+0000", 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003456822 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449945+0000", 2024-09-18T00:59:50.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0896000000000003e-05 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449971+0000", 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5996999999999999e-05 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449986+0000", 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.49e-05 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579978+0000", 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.129992723 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580157+0000", 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179006 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580379+0000", 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022127099999999999 2024-09-18T00:59:50.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98073 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4696064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.243930712000001, 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13113503100000001, 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:50.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98073 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450057+0000", 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450060+0000", 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5329999999999999e-06 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450072+0000", 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1919999999999999e-05 2024-09-18T00:59:50.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450080+0000", 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9500000000000001e-06 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580951+0000", 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.130871128 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581057+0000", 2024-09-18T00:59:50.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010501399999999999 2024-09-18T00:59:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581191+0000", 2024-09-18T00:59:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013479900000000001 2024-09-18T00:59:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:50.805 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:50.805 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:50.806 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:50.806 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:50.713+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:50.814 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:50.813+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:50.915 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:50.913+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:50.938 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:50.938 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:50.938 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:50.938 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:50.952 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:51.016 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:51.013+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:51.117 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:51.113+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:51.121 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:51.121 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:51.121 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:51.121 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:51.121 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:51.135 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:51.218 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:51.217+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:51.260 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:51.260 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:51.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:51.260 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:51.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.260 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:51.261 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:59:51.261 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 122.483784464, 2024-09-18T00:59:51.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:59:51.261 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.261 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.261 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:51.261 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.261 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.261 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.261 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:59:51.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:59:51.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:59:51.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:59:51.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:59:51.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:59:51.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:59:51.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:51.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:59:51.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:59:51.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:59:51.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:59:51.267 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.267 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.267 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.268 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87406 3.19 3:9d734bda:::rbd_data.117c8934add3.000000000000007c:head [stat,delete] snapc 1f=[1f] RETRY=8 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:59:51.268 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:59:51.268 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.835861342000001, 2024-09-18T00:59:51.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097024104999999999, 2024-09-18T00:59:51.268 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.268 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.268 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:51.269 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.269 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.269 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.269 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87406 2024-09-18T00:59:51.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.269 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:59:51.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425532+0000", 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027458309999999998 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425537+0000", 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2839999999999997e-06 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425551+0000", 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3682999999999999e-05 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425559+0000", 2024-09-18T00:59:51.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5329999999999997e-06 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515035+0000", 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.089476411000000006 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515168+0000", 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013309699999999999 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515475+0000", 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030662199999999999 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519553+0000", 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040781259999999996 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:51.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519665+0000", 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112669 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519748+0000", 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2270999999999999e-05 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519810+0000", 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2577999999999996e-05 2024-09-18T00:59:51.274 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.275 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.275 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.275 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87414 3.19 3:9c145b25:::rbd_data.117c8934add3.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:59:51.275 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:51.275 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.833037803000003, 2024-09-18T00:59:51.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095886693999999995, 2024-09-18T00:59:51.275 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.275 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.276 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:51.276 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.276 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.276 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.276 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87414 2024-09-18T00:59:51.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.276 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:51.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:51.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:51.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2024-09-18T00:59:51.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425618+0000", 2024-09-18T00:59:51.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9810000000000003e-06 2024-09-18T00:59:51.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425625+0000", 2024-09-18T00:59:51.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0870000000000001e-06 2024-09-18T00:59:51.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425630+0000", 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.262e-06 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518417+0000", 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092787816999999995 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518593+0000", 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017569100000000001 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518939+0000", 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034572299999999998 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521320+0000", 2024-09-18T00:59:51.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023809619999999999 2024-09-18T00:59:51.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:51.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521343+0000", 2024-09-18T00:59:51.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3317000000000001e-05 2024-09-18T00:59:51.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:51.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521422+0000", 2024-09-18T00:59:51.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480999999999996e-05 2024-09-18T00:59:51.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521496+0000", 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4909999999999999e-05 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99159 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 393216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369266+0000", 2024-09-18T00:59:51.282 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.889381452999999, 2024-09-18T00:59:51.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096008205999999999, 2024-09-18T00:59:51.283 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.283 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.283 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.283 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.283 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.283 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.283 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99159 2024-09-18T00:59:51.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.283 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T00:59:51.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T00:59:51.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369409+0000", 2024-09-18T00:59:51.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014342599999999999 2024-09-18T00:59:51.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369415+0000", 2024-09-18T00:59:51.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.31e-06 2024-09-18T00:59:51.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369426+0000", 2024-09-18T00:59:51.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1677e-05 2024-09-18T00:59:51.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369435+0000", 2024-09-18T00:59:51.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2024-09-18T00:59:51.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.464949+0000", 2024-09-18T00:59:51.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095513718999999997 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465099+0000", 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014984300000000001 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465274+0000", 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017554499999999999 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.287 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99160 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2336768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369451+0000", 2024-09-18T00:59:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.889196884, 2024-09-18T00:59:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096064584999999994, 2024-09-18T00:59:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.288 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.289 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.289 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.289 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99160 2024-09-18T00:59:51.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.289 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T00:59:51.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369564+0000", 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113309 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369569+0000", 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5269999999999998e-06 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369582+0000", 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2952999999999999e-05 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369592+0000", 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0939999999999997e-06 2024-09-18T00:59:51.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465328+0000", 2024-09-18T00:59:51.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095736285000000004 2024-09-18T00:59:51.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465403+0000", 2024-09-18T00:59:51.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5498000000000001e-05 2024-09-18T00:59:51.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465515+0000", 2024-09-18T00:59:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111919 2024-09-18T00:59:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99161 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 413696~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369608+0000", 2024-09-18T00:59:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.889039255, 2024-09-18T00:59:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096302270999999995, 2024-09-18T00:59:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99161 2024-09-18T00:59:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373792+0000", 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004184063 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373801+0000", 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3729999999999993e-06 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373818+0000", 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7280000000000001e-05 2024-09-18T00:59:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373829+0000", 2024-09-18T00:59:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0540000000000001e-05 2024-09-18T00:59:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465596+0000", 2024-09-18T00:59:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.091767910999999994 2024-09-18T00:59:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465740+0000", 2024-09-18T00:59:51.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143683 2024-09-18T00:59:51.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465911+0000", 2024-09-18T00:59:51.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000170421 2024-09-18T00:59:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99172 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2459648~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.884443912000002, 2024-09-18T00:59:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095842035000000006, 2024-09-18T00:59:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99172 2024-09-18T00:59:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9299999999999997e-07 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374207+0000", 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8899999999999999e-06 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374216+0000", 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3669999999999997e-06 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374223+0000", 2024-09-18T00:59:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9759999999999998e-06 2024-09-18T00:59:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469763+0000", 2024-09-18T00:59:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095540307000000005 2024-09-18T00:59:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469894+0000", 2024-09-18T00:59:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013135600000000001 2024-09-18T00:59:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470046+0000", 2024-09-18T00:59:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015144600000000001 2024-09-18T00:59:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99173 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2480128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374231+0000", 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.884416352000002, 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096195265000000002, 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99173 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374232+0000", 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4600000000000004e-07 2024-09-18T00:59:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374235+0000", 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8569999999999999e-06 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374243+0000", 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5930000000000006e-06 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374250+0000", 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9330000000000004e-06 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470113+0000", 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095862952000000001 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470259+0000", 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014534300000000001 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470426+0000", 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016784100000000001 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99174 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2500608~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374259+0000", 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.884388045999998, 2024-09-18T00:59:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096540567999999993, 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99174 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374260+0000", 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2600000000000002e-07 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374263+0000", 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8770000000000002e-06 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374272+0000", 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6070000000000008e-06 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374279+0000", 2024-09-18T00:59:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8430000000000004e-06 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470492+0000", 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096213513000000001 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470632+0000", 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013954899999999999 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470800+0000", 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016845299999999999 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99175 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 536576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374287+0000", 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.884360149999999, 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096867154999999996, 2024-09-18T00:59:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99175 2024-09-18T00:59:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T00:59:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T00:59:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374288+0000", 2024-09-18T00:59:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9999999999999997e-07 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374291+0000", 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.92e-06 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374300+0000", 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5939999999999994e-06 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374307+0000", 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9829999999999999e-06 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470858+0000", 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096551085999999994 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470990+0000", 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013223900000000001 2024-09-18T00:59:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471155+0000", 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016463299999999999 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99176 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2521088~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374315+0000", 2024-09-18T00:59:51.320 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.884332200000003, 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097204272999999994, 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99176 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T00:59:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T00:59:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374316+0000", 2024-09-18T00:59:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1999999999999999e-07 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374336+0000", 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9879999999999999e-05 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374346+0000", 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0353999999999999e-05 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374354+0000", 2024-09-18T00:59:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7130000000000005e-06 2024-09-18T00:59:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471210+0000", 2024-09-18T00:59:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096856271999999993 2024-09-18T00:59:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471350+0000", 2024-09-18T00:59:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013940900000000001 2024-09-18T00:59:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471520+0000", 2024-09-18T00:59:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000169925 2024-09-18T00:59:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99177 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 557056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374364+0000", 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.884283336999999, 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097507736999999997, 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99177 2024-09-18T00:59:51.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.327 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T00:59:51.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T00:59:51.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377919+0000", 2024-09-18T00:59:51.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035553070000000002 2024-09-18T00:59:51.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377927+0000", 2024-09-18T00:59:51.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4569999999999999e-06 2024-09-18T00:59:51.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377943+0000", 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5563e-05 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377953+0000", 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0413e-05 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471578+0000", 2024-09-18T00:59:51.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093625036999999994 2024-09-18T00:59:51.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471710+0000", 2024-09-18T00:59:51.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013170900000000001 2024-09-18T00:59:51.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471872+0000", 2024-09-18T00:59:51.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162251 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99183 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 618496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378152+0000", 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.880495301000003, 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095979540000000002, 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.331 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.332 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99183 2024-09-18T00:59:51.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.332 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T00:59:51.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T00:59:51.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378153+0000", 2024-09-18T00:59:51.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.23e-07 2024-09-18T00:59:51.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378156+0000", 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9340000000000002e-06 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378164+0000", 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4630000000000001e-06 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378173+0000", 2024-09-18T00:59:51.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8130000000000002e-06 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473851+0000", 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678257000000003 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473982+0000", 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000130875 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474132+0000", 2024-09-18T00:59:51.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014947499999999999 2024-09-18T00:59:51.336 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.336 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.336 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.336 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99184 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2603008~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.336 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378209+0000", 2024-09-18T00:59:51.336 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.880438759, 2024-09-18T00:59:51.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096289871999999999, 2024-09-18T00:59:51.337 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.337 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.337 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.337 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.337 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.337 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.337 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99184 2024-09-18T00:59:51.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.337 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378210+0000", 2024-09-18T00:59:51.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2999999999999999e-07 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378213+0000", 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378223+0000", 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378230+0000", 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5830000000000001e-06 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474192+0000", 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961742000000003 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474324+0000", 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000132099 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474499+0000", 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174551 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.341 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99185 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 638976~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.342 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378243+0000", 2024-09-18T00:59:51.342 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.880404331999998, 2024-09-18T00:59:51.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096618187999999994, 2024-09-18T00:59:51.342 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.342 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.342 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.342 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.342 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.342 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.342 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99185 2024-09-18T00:59:51.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378244+0000", 2024-09-18T00:59:51.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5700000000000001e-07 2024-09-18T00:59:51.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378248+0000", 2024-09-18T00:59:51.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5130000000000001e-06 2024-09-18T00:59:51.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378256+0000", 2024-09-18T00:59:51.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8429999999999999e-06 2024-09-18T00:59:51.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378263+0000", 2024-09-18T00:59:51.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9870000000000002e-06 2024-09-18T00:59:51.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474554+0000", 2024-09-18T00:59:51.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096290693999999996 2024-09-18T00:59:51.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474692+0000", 2024-09-18T00:59:51.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013761900000000001 2024-09-18T00:59:51.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474861+0000", 2024-09-18T00:59:51.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016967499999999999 2024-09-18T00:59:51.347 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.347 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.347 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.347 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99186 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 659456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.347 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:51.347 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.880374781999997, 2024-09-18T00:59:51.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096958847000000001, 2024-09-18T00:59:51.348 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.348 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.348 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.348 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.348 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.348 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.348 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99186 2024-09-18T00:59:51.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.348 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:51.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:51.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1699999999999997e-07 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378277+0000", 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0800000000000002e-06 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378285+0000", 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5699999999999993e-06 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378292+0000", 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9600000000000003e-06 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474924+0000", 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096632207999999997 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475067+0000", 2024-09-18T00:59:51.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143175 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475232+0000", 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016413699999999999 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99187 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 679936~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.352 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378301+0000", 2024-09-18T00:59:51.353 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.880346508999999, 2024-09-18T00:59:51.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097293710000000005, 2024-09-18T00:59:51.353 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.353 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.353 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.353 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.353 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.353 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.353 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99187 2024-09-18T00:59:51.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.353 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T00:59:51.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T00:59:51.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378302+0000", 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378305+0000", 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.066e-06 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378315+0000", 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0443e-05 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378323+0000", 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3239999999999997e-06 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475291+0000", 2024-09-18T00:59:51.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096968726000000005 2024-09-18T00:59:51.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475430+0000", 2024-09-18T00:59:51.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013847499999999999 2024-09-18T00:59:51.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475595+0000", 2024-09-18T00:59:51.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016497899999999999 2024-09-18T00:59:51.358 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.358 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.358 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.358 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99188 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 700416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.358 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378332+0000", 2024-09-18T00:59:51.358 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.880315672999998, 2024-09-18T00:59:51.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097626477000000003, 2024-09-18T00:59:51.358 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.358 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99188 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381945+0000", 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036127569999999999 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381960+0000", 2024-09-18T00:59:51.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.554e-05 2024-09-18T00:59:51.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381977+0000", 2024-09-18T00:59:51.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6733000000000001e-05 2024-09-18T00:59:51.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381987+0000", 2024-09-18T00:59:51.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.031e-05 2024-09-18T00:59:51.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475651+0000", 2024-09-18T00:59:51.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093664175000000002 2024-09-18T00:59:51.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475789+0000", 2024-09-18T00:59:51.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000137181 2024-09-18T00:59:51.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475958+0000", 2024-09-18T00:59:51.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016978100000000001 2024-09-18T00:59:51.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.363 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99195 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 761856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.876442867999998, 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961833999999996, 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99195 2024-09-18T00:59:51.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:51.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T00:59:51.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382208+0000", 2024-09-18T00:59:51.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0469999999999998e-06 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382217+0000", 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5159999999999994e-06 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382224+0000", 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.64e-06 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.477889+0000", 2024-09-18T00:59:51.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095664955999999995 2024-09-18T00:59:51.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478012+0000", 2024-09-18T00:59:51.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012299899999999999 2024-09-18T00:59:51.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478166+0000", 2024-09-18T00:59:51.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015497899999999999 2024-09-18T00:59:51.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.369 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.369 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99196 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 88064~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:51.369 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382233+0000", 2024-09-18T00:59:51.369 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.876414938000003, 2024-09-18T00:59:51.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096312801000000003, 2024-09-18T00:59:51.370 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:51.370 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:51.370 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:51.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:51.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:51.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:51.370 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99196 2024-09-18T00:59:51.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.371 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:51.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:51.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T00:59:51.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:51.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T00:59:51.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:51.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:51.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385636+0000", 2024-09-18T00:59:51.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034037450000000001 2024-09-18T00:59:51.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:51.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385642+0000", 2024-09-18T00:59:51.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9900000000000002e-06 2024-09-18T00:59:51.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:51.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385659+0000", 2024-09-18T00:59:51.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6753e-05 2024-09-18T00:59:51.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:51.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385669+0000", 2024-09-18T00:59:51.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9170000000000001e-06 2024-09-18T00:59:51.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478225+0000", 2024-09-18T00:59:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092555514000000005 2024-09-18T00:59:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478366+0000", 2024-09-18T00:59:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014198699999999999 2024-09-18T00:59:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478545+0000", 2024-09-18T00:59:51.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017889499999999999 2024-09-18T00:59:51.375 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.375 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.375 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.375 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:51.375 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:51.375 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:51.377 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T00:59:51.378 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:51.378 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:51.373+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:51.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:51.478 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:51.477+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:51.564 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:51.564 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:51.564 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:51.564 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:51.579 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:51.579 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:51.577+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:51.587 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:51.587 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:51.604 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:51.604 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:51.620 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:51.621 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:51.637 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:51.637 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:51.655 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:51.655 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:51.673 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:51.673 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:51.680 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:51.681+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:51.693 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:51.693 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:51.714 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:51.714 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:51.742 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:51.742 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:51.757 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:51.801 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:51.801+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:51.882 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:51.881+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:51.923 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:51.923 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:51.937 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:51.982 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:51.981+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:52.083 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:52.081+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:52.102 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:52.102 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:52.116 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:52.208 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:52.209+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:52.280 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:52.280 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:52.285 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:52.285+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:52.294 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:52.386 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:52.385+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:52.469 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:52.469 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:52.487 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:52.487 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:52.485+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:52.587 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:52.585+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:52.664 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:52.664 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:52.678 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:52.724 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:52.725+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:52.790 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:52.789+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:52.799 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:52.800 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:52.814 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:52.891 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:52.889+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:52.982 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:52.982 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:52.992 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:52.989+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:52.997 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:53.093 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:53.093+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:53.131 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:53.131 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:53.146 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:53.194 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:53.193+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:53.266 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:53.266 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:53.281 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:53.295 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:53.293+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:53.379 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:59:53.426 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:53.425+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:53.459 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:53.460 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:53.476 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:53.497 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:53.497+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:53.597 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:53.597+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:53.636 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:53.636 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:53.652 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:53.698 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:53.697+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:53.733 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:53.777 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:53.777 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:53.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:53.798 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:53.797+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:53.881 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:53.899 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:53.899 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:53.899 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:53.897+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:53.916 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:53.934 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:53.935 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:53.955 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:53.955 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.665279715, 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:53.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T00:59:53.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T00:59:53.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T00:59:53.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.963 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.468917225, 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:53.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.967 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.468821898, 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T00:59:53.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T00:59:53.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:53.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.468796459, 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:53.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T00:59:53.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.977 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:53.977 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:53.977 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.46877393600001, 2024-09-18T00:59:53.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T00:59:53.977 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:53.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:53.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:53.977 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:53.977 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:53.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:53.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T00:59:53.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.978 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:53.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:53.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:53.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:53.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T00:59:53.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.468750496, 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:53.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:53.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T00:59:53.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T00:59:53.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:53.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T00:59:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T00:59:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T00:59:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T00:59:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:53.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.468726686, 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:53.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:53.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T00:59:53.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:53.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T00:59:53.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:59:53.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:53.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T00:59:53.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T00:59:53.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:53.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T00:59:53.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T00:59:53.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T00:59:53.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.468704321, 2024-09-18T00:59:53.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T00:59:53.993 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:53.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:53.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:53.993 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:53.993 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:53.993 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:53.993 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T00:59:53.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.993 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:53.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T00:59:53.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:53.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T00:59:53.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T00:59:53.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:53.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T00:59:53.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T00:59:53.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T00:59:53.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.468677704, 2024-09-18T00:59:53.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:53.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T00:59:53.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:53.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:53.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:53.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T00:59:53.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T00:59:53.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:53.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T00:59:54.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T00:59:54.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T00:59:54.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7499 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2064384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.46450519, 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045607969999999998, 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7499 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490094+0000", 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490095+0000", 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.03e-07 2024-09-18T00:59:54.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490103+0000", 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.174e-06 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490109+0000", 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4770000000000002e-06 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490112+0000", 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4910000000000001e-06 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535558+0000", 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045446161999999998 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535636+0000", 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7311000000000006e-05 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535702+0000", 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6952000000000005e-05 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.464459625, 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.007 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T00:59:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T00:59:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.46443328399999, 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T00:59:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T00:59:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.464410614, 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T00:59:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T00:59:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.464392241, 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T00:59:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T00:59:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T00:59:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T00:59:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T00:59:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T00:59:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T00:59:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:59:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T00:59:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T00:59:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T00:59:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T00:59:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T00:59:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T00:59:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.464372453, 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T00:59:54.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T00:59:54.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T00:59:54.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T00:59:54.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T00:59:54.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.464345583, 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T00:59:54.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.033 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T00:59:54.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T00:59:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 78.388155114, 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T00:59:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T00:59:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.977090017999998, 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T00:59:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T00:59:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T00:59:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T00:59:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:140635 3.d 3:b08054e4:::rbd_object_map.117ced55205b.0000000000000023:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.615976623999998, 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86222047999999996, 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 140635 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338625+0000", 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338634+0000", 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9099999999999994e-06 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338657+0000", 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3672999999999998e-05 2024-09-18T00:59:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338675+0000", 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7589999999999999e-05 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338767+0000", 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1896000000000002e-05 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T00:59:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338776+0000", 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.38e-06 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200213+0000", 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86143668200000001 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200432+0000", 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021884199999999999 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200844+0000", 2024-09-18T00:59:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000411887 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:155664 3.d 3:b496760f:::rbd_data.117c1d78ece1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:40.905926+0000", 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.048673519999999, 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046053647000000003, 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 155664 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905927+0000", 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.175e-06 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909267+0000", 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003339266 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909287+0000", 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605e-05 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909297+0000", 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909363+0000", 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6437000000000001e-05 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909483+0000", 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119861 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T00:59:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909868+0000", 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038564099999999999 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.917828+0000", 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079592570000000008 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951788+0000", 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033960314999999998 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951895+0000", 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107316 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951980+0000", 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4460000000000001e-05 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.060 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:54.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:54.062 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:54.063 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:54.063 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:54.064 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:54.064 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:54.065 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:54.066 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:54.061+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:54.162 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:54.161+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:54.229 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:54.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:54.230 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:54.230 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:54.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:54.286 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:54.285+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:54.357 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T00:59:54.364 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:54.361+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:54.422 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:54.422 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:54.422 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:54.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:54.465 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:54.465+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.47890982499999, 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:59:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.477980234, 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:59:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:59:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.47796617399999, 2024-09-18T00:59:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:59:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.476675599, 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:59:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.47665512699999, 2024-09-18T00:59:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:59:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:59:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.47386067400001, 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:59:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.47384549, 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:59:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:59:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:59:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:59:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:59:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:59:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:59:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:59:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:59:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:59:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.473822544, 2024-09-18T00:59:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:59:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:59:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:59:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:59:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:59:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:59:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:59:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:59:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:59:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:59:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:59:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:59:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:59:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:59:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:59:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:59:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.469644905, 2024-09-18T00:59:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:59:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:59:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:59:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:59:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:59:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:59:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:59:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:59:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:59:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:59:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:59:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.46962368800001, 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:59:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:59:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.469602305, 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:59:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:59:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:59:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:59:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:59:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:59:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.465702451, 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:59:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.46568825, 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:59:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:59:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.46567021300001, 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.465656147, 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:59:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:59:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:59:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:59:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:59:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.46163079600001, 2024-09-18T00:59:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:59:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:59:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:59:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.461616784, 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:59:54.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.46160282, 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.652 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:59:54.652 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:54.652 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.699744866, 2024-09-18T00:59:54.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:59:54.652 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:54.652 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:59:54.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:59:54.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.574268495, 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.657 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:54.662 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:54.664 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:54.664 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:54.661+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:54.765 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:54.765+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:54.839 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:54.882 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:54.882 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:54.883 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:54.884 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:54.881+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:54.984 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:54.985+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:55.011 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:55.011 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:55.011 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:55.011 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:55.011 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:55.025 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:55.085 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:55.081+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:55.147 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98052 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4179968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.704485159999997, 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131360583, 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98052 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440037+0000", 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440040+0000", 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7159999999999999e-06 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440048+0000", 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7169999999999997e-06 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440056+0000", 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1860000000000003e-06 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.570989+0000", 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13093384999999999 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571175+0000", 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018504799999999999 2024-09-18T00:59:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571397+0000", 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022245899999999999 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98053 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4466688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.704457290999997, 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13182195299999999, 2024-09-18T00:59:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98053 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440065+0000", 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440068+0000", 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6699999999999998e-06 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440076+0000", 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7069999999999998e-06 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440083+0000", 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9929999999999998e-06 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571475+0000", 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13139158100000001 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571657+0000", 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018247099999999999 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571886+0000", 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022892400000000001 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98054 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2113536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.704429431000001, 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13228911600000001, 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98054 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440093+0000", 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440096+0000", 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440105+0000", 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7739999999999993e-06 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440112+0000", 2024-09-18T00:59:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2130000000000002e-06 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571970+0000", 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13185812199999999 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572159+0000", 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000188665 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572381+0000", 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022273200000000001 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98055 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2400256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.704399250999998, 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273966600000001, 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98055 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440123+0000", 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440126+0000", 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440137+0000", 2024-09-18T00:59:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0543000000000001e-05 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440144+0000", 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572457+0000", 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132312807 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572636+0000", 2024-09-18T00:59:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882299999999999 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572862+0000", 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022645200000000001 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98056 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2686976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.704368674000001, 2024-09-18T00:59:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13308801200000001, 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98056 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440154+0000", 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5300000000000004e-07 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440156+0000", 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.903e-06 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440165+0000", 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7600000000000008e-06 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440174+0000", 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6969999999999999e-06 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572960+0000", 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132785984 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573087+0000", 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012728699999999999 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573241+0000", 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153728 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98057 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2973696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.704339081000001, 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13357376900000001, 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98057 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440183+0000", 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440186+0000", 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8700000000000001e-06 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440195+0000", 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7099999999999996e-06 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440202+0000", 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2429999999999999e-06 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573338+0000", 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13313578400000001 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573526+0000", 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018780399999999999 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573756+0000", 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230755 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98058 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.704311197999999, 2024-09-18T00:59:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13387749400000001, 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98058 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440627+0000", 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041660699999999999 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440634+0000", 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.596e-06 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440650+0000", 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6164e-05 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440660+0000", 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.023e-05 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573833+0000", 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13317317000000001 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573945+0000", 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111731 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574088+0000", 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000142996 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98059 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3547136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.703843851000002, 2024-09-18T00:59:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13391484000000001, 2024-09-18T00:59:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98059 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441760+0000", 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001082185 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441780+0000", 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0463e-05 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441829+0000", 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8763000000000001e-05 2024-09-18T00:59:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441849+0000", 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9423e-05 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574173+0000", 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13232460400000001 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574367+0000", 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019403000000000001 2024-09-18T00:59:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574593+0000", 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000225372 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98060 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3833856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.70264177, 2024-09-18T00:59:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13320725899999999, 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98060 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441881+0000", 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6160000000000001e-06 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441889+0000", 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8439999999999998e-06 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441909+0000", 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9856000000000001e-05 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441925+0000", 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6039999999999999e-05 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574669+0000", 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13274412799999999 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574860+0000", 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019038599999999999 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575087+0000", 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022738899999999999 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98061 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4120576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.702575013999997, 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13362664099999999, 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98061 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441948+0000", 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441954+0000", 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3099999999999997e-06 2024-09-18T00:59:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441981+0000", 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7095999999999999e-05 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441998+0000", 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6237000000000001e-05 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575165+0000", 2024-09-18T00:59:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13316721200000001 2024-09-18T00:59:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575355+0000", 2024-09-18T00:59:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000190218 2024-09-18T00:59:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575573+0000", 2024-09-18T00:59:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021824799999999999 2024-09-18T00:59:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98062 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4407296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.702503045, 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13404179099999999, 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98062 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445789+0000", 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037705569999999999 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445805+0000", 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.558e-05 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445846+0000", 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445871+0000", 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5072999999999999e-05 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575651+0000", 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12977999600000001 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575835+0000", 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000183931 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576060+0000", 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022558800000000001 2024-09-18T00:59:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98064 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2115584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.698538606, 2024-09-18T00:59:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13103894799999999, 2024-09-18T00:59:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98064 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445985+0000", 2024-09-18T00:59:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-18T00:59:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445991+0000", 2024-09-18T00:59:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446014+0000", 2024-09-18T00:59:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3056000000000001e-05 2024-09-18T00:59:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446030+0000", 2024-09-18T00:59:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5662999999999999e-05 2024-09-18T00:59:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576622+0000", 2024-09-18T00:59:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13059273900000001 2024-09-18T00:59:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576805+0000", 2024-09-18T00:59:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018304899999999999 2024-09-18T00:59:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577022+0000", 2024-09-18T00:59:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216481 2024-09-18T00:59:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98065 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2402304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.698471490000003, 2024-09-18T00:59:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131457463, 2024-09-18T00:59:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98065 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446051+0000", 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1260000000000001e-06 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446057+0000", 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2040000000000002e-06 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446077+0000", 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446092+0000", 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5124e-05 2024-09-18T00:59:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577098+0000", 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13100595100000001 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577284+0000", 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000186152 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577508+0000", 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022314000000000001 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98066 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2689024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.698409284, 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13189522100000001, 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98066 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446113+0000", 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0240000000000001e-06 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446120+0000", 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446137+0000", 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7479e-05 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446152+0000", 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4633e-05 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577587+0000", 2024-09-18T00:59:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13143572100000001 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577786+0000", 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019872999999999999 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578008+0000", 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022141399999999999 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98067 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2975744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.698351621, 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132319567, 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.217 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98067 2024-09-18T00:59:55.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.217 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:55.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446171+0000", 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446178+0000", 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699999999999997e-06 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446196+0000", 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7989999999999999e-05 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446213+0000", 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7493999999999999e-05 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578083+0000", 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13186995000000001 2024-09-18T00:59:55.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578266+0000", 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018321099999999999 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578490+0000", 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022316600000000001 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98068 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3262464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.698290714999999, 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273661, 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98068 2024-09-18T00:59:55.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.222 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:55.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:55.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446232+0000", 2024-09-18T00:59:55.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.187e-06 2024-09-18T00:59:55.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446238+0000", 2024-09-18T00:59:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446257+0000", 2024-09-18T00:59:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:55.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446272+0000", 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5532999999999998e-05 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578569+0000", 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132296631 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578749+0000", 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017978300000000001 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578967+0000", 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218736 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98069 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3549184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.226 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.698231042000003, 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13311771, 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98069 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446292+0000", 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446298+0000", 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4470000000000003e-06 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446340+0000", 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1705999999999997e-05 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446358+0000", 2024-09-18T00:59:55.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7873999999999999e-05 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579072+0000", 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13271396399999999 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579213+0000", 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014133900000000001 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579408+0000", 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019492700000000001 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98070 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3835904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.698137598999999, 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133517145, 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.231 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98070 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446385+0000", 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446393+0000", 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2629999999999997e-06 2024-09-18T00:59:55.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446413+0000", 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9930000000000001e-05 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446428+0000", 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5336e-05 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579494+0000", 2024-09-18T00:59:55.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13306637399999999 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579682+0000", 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018740900000000001 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579901+0000", 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021946300000000001 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98071 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.698074433000002, 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133931615, 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98071 2024-09-18T00:59:55.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449904+0000", 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003456822 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449945+0000", 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0896000000000003e-05 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449971+0000", 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5996999999999999e-05 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449986+0000", 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.49e-05 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579978+0000", 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.129992723 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580157+0000", 2024-09-18T00:59:55.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179006 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580379+0000", 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022127099999999999 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98073 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4696064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.694465305999998, 2024-09-18T00:59:55.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13113503100000001, 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98073 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:55.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450057+0000", 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450060+0000", 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5329999999999999e-06 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450072+0000", 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1919999999999999e-05 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450080+0000", 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9500000000000001e-06 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580951+0000", 2024-09-18T00:59:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.130871128 2024-09-18T00:59:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581057+0000", 2024-09-18T00:59:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010501399999999999 2024-09-18T00:59:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:55.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581191+0000", 2024-09-18T00:59:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013479900000000001 2024-09-18T00:59:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:55.245 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:55.245 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:55.246 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:55.247 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:55.245+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:55.348 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:55.345+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:159824 3.3 3:c3eb3f9e:::rbd_header.117cb1e63749:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:55.387397+0000", 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.0030814369999999998, 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0031110629999999998, 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "waiting for sub ops", 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.391 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.391 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 159824 2024-09-18T00:59:55.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.391 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:55.387397+0000", 2024-09-18T00:59:55.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:55.387397+0000", 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:55.387399+0000", 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6429999999999999e-06 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:55.387413+0000", 2024-09-18T00:59:55.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3339999999999999e-05 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:55.387443+0000", 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0673000000000003e-05 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:55.387463+0000", 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0100000000000001e-05 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:55.387555+0000", 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1509999999999996e-05 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:55.387645+0000", 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0421999999999996e-05 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:55.387831+0000", 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185804 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:55.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:55.390008+0000", 2024-09-18T00:59:55.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002177305 2024-09-18T00:59:55.395 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.395 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.395 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.395 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.395 INFO:teuthology.orchestra.run.smithi195.stdout: ], 2024-09-18T00:59:55.395 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 1 2024-09-18T00:59:55.395 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:55.410 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:55.448 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:55.445+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:55.532 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:55.532 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:55.532 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:55.547 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:55.549 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:55.549+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:55.650 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:55.649+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:55.669 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 126.89181623499999, 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T00:59:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87406 3.19 3:9d734bda:::rbd_data.117c8934add3.000000000000007c:head [stat,delete] snapc 1f=[1f] RETRY=8 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 67.243893112999999, 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097024104999999999, 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87406 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425532+0000", 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027458309999999998 2024-09-18T00:59:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425537+0000", 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2839999999999997e-06 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425551+0000", 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3682999999999999e-05 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425559+0000", 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5329999999999997e-06 2024-09-18T00:59:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515035+0000", 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.089476411000000006 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515168+0000", 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013309699999999999 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515475+0000", 2024-09-18T00:59:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030662199999999999 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519553+0000", 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040781259999999996 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519665+0000", 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112669 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519748+0000", 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2270999999999999e-05 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519810+0000", 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2577999999999996e-05 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87414 3.19 3:9c145b25:::rbd_data.117c8934add3.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 67.241069573999994, 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095886693999999995, 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87414 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425618+0000", 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9810000000000003e-06 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425625+0000", 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0870000000000001e-06 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425630+0000", 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.262e-06 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518417+0000", 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092787816999999995 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518593+0000", 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017569100000000001 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518939+0000", 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034572299999999998 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521320+0000", 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023809619999999999 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521343+0000", 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3317000000000001e-05 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521422+0000", 2024-09-18T00:59:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480999999999996e-05 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521496+0000", 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4909999999999999e-05 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99159 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 393216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369266+0000", 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.297413224000003, 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096008205999999999, 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99159 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369409+0000", 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014342599999999999 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369415+0000", 2024-09-18T00:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.31e-06 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369426+0000", 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1677e-05 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369435+0000", 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.464949+0000", 2024-09-18T00:59:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095513718999999997 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465099+0000", 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014984300000000001 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465274+0000", 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017554499999999999 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99160 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2336768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369451+0000", 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.297228654999998, 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096064584999999994, 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99160 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369564+0000", 2024-09-18T00:59:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113309 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369569+0000", 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5269999999999998e-06 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369582+0000", 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2952999999999999e-05 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369592+0000", 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0939999999999997e-06 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465328+0000", 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095736285000000004 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465403+0000", 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5498000000000001e-05 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465515+0000", 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111919 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99161 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 413696~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369608+0000", 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.297071025999998, 2024-09-18T00:59:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096302270999999995, 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99161 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373792+0000", 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004184063 2024-09-18T00:59:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373801+0000", 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3729999999999993e-06 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373818+0000", 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7280000000000001e-05 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373829+0000", 2024-09-18T00:59:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0540000000000001e-05 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465596+0000", 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.091767910999999994 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465740+0000", 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143683 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465911+0000", 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000170421 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99172 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2459648~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.292475682999999, 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095842035000000006, 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99172 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T00:59:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9299999999999997e-07 2024-09-18T00:59:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374207+0000", 2024-09-18T00:59:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8899999999999999e-06 2024-09-18T00:59:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374216+0000", 2024-09-18T00:59:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3669999999999997e-06 2024-09-18T00:59:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374223+0000", 2024-09-18T00:59:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9759999999999998e-06 2024-09-18T00:59:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469763+0000", 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095540307000000005 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469894+0000", 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013135600000000001 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470046+0000", 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015144600000000001 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99173 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2480128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374231+0000", 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.292448123, 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096195265000000002, 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99173 2024-09-18T00:59:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T00:59:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T00:59:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374232+0000", 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4600000000000004e-07 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374235+0000", 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8569999999999999e-06 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374243+0000", 2024-09-18T00:59:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5930000000000006e-06 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374250+0000", 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9330000000000004e-06 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470113+0000", 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095862952000000001 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470259+0000", 2024-09-18T00:59:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014534300000000001 2024-09-18T00:59:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470426+0000", 2024-09-18T00:59:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016784100000000001 2024-09-18T00:59:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99174 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2500608~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374259+0000", 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.292419817000003, 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096540567999999993, 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99174 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T00:59:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T00:59:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374260+0000", 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2600000000000002e-07 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374263+0000", 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8770000000000002e-06 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374272+0000", 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6070000000000008e-06 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374279+0000", 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8430000000000004e-06 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470492+0000", 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096213513000000001 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470632+0000", 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013954899999999999 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470800+0000", 2024-09-18T00:59:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016845299999999999 2024-09-18T00:59:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99175 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 536576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374287+0000", 2024-09-18T00:59:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.292391920999997, 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096867154999999996, 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99175 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374288+0000", 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9999999999999997e-07 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374291+0000", 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.92e-06 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374300+0000", 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5939999999999994e-06 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374307+0000", 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9829999999999999e-06 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470858+0000", 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096551085999999994 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470990+0000", 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013223900000000001 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471155+0000", 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016463299999999999 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99176 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2521088~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374315+0000", 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.292363971, 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097204272999999994, 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99176 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374316+0000", 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1999999999999999e-07 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374336+0000", 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9879999999999999e-05 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374346+0000", 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0353999999999999e-05 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374354+0000", 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7130000000000005e-06 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471210+0000", 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096856271999999993 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471350+0000", 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013940900000000001 2024-09-18T00:59:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471520+0000", 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000169925 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99177 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 557056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374364+0000", 2024-09-18T00:59:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.292315107999997, 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097507736999999997, 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99177 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T00:59:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377919+0000", 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035553070000000002 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377927+0000", 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4569999999999999e-06 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377943+0000", 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5563e-05 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377953+0000", 2024-09-18T00:59:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0413e-05 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471578+0000", 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093625036999999994 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471710+0000", 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013170900000000001 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471872+0000", 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162251 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99183 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 618496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378152+0000", 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.288527072000001, 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095979540000000002, 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99183 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378153+0000", 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.23e-07 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378156+0000", 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9340000000000002e-06 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378164+0000", 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4630000000000001e-06 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378173+0000", 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8130000000000002e-06 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473851+0000", 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678257000000003 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473982+0000", 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000130875 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474132+0000", 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014947499999999999 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99184 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2603008~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378209+0000", 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.288470529999998, 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096289871999999999, 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99184 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378210+0000", 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2999999999999999e-07 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378213+0000", 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378223+0000", 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378230+0000", 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5830000000000001e-06 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474192+0000", 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961742000000003 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474324+0000", 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000132099 2024-09-18T00:59:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474499+0000", 2024-09-18T00:59:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174551 2024-09-18T00:59:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99185 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 638976~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378243+0000", 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.288436103000002, 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096618187999999994, 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99185 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T00:59:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T00:59:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378244+0000", 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5700000000000001e-07 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378248+0000", 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5130000000000001e-06 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378256+0000", 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8429999999999999e-06 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378263+0000", 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9870000000000002e-06 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474554+0000", 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096290693999999996 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474692+0000", 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013761900000000001 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474861+0000", 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016967499999999999 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99186 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 659456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.288406553000002, 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096958847000000001, 2024-09-18T00:59:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99186 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1699999999999997e-07 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378277+0000", 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0800000000000002e-06 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378285+0000", 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5699999999999993e-06 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378292+0000", 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9600000000000003e-06 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474924+0000", 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096632207999999997 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475067+0000", 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143175 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475232+0000", 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016413699999999999 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99187 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 679936~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378301+0000", 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.288378280000003, 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097293710000000005, 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99187 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378302+0000", 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378305+0000", 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.066e-06 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378315+0000", 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0443e-05 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378323+0000", 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3239999999999997e-06 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475291+0000", 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096968726000000005 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475430+0000", 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013847499999999999 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475595+0000", 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016497899999999999 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99188 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 700416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378332+0000", 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.288347444000003, 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097626477000000003, 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99188 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381945+0000", 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036127569999999999 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381960+0000", 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.554e-05 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381977+0000", 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6733000000000001e-05 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381987+0000", 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.031e-05 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475651+0000", 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093664175000000002 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475789+0000", 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000137181 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475958+0000", 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016978100000000001 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99195 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 761856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.284474639000003, 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961833999999996, 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99195 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382208+0000", 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0469999999999998e-06 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382217+0000", 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5159999999999994e-06 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382224+0000", 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.64e-06 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.477889+0000", 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095664955999999995 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478012+0000", 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012299899999999999 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478166+0000", 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015497899999999999 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99196 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 88064~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:59:55.761 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382233+0000", 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.284446709000001, 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096312801000000003, 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99196 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T00:59:55.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385636+0000", 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034037450000000001 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385642+0000", 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9900000000000002e-06 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385659+0000", 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6753e-05 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385669+0000", 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9170000000000001e-06 2024-09-18T00:59:55.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478225+0000", 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092555514000000005 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478366+0000", 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014198699999999999 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:55.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:55.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478545+0000", 2024-09-18T00:59:55.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017889499999999999 2024-09-18T00:59:55.766 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.766 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.766 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.766 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:55.766 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:55.766 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:55.766 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6] out_osds: [2, 3, 0, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:59:55.766 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:59:55.766 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T00:59:55.767 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-18T00:59:55.769 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:59:55.769 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:55.769+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:55.868 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:55.869+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:55.940 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:55.940 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:55.941 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:55.941 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:55.955 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:59:55.969 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:55.965+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:56.066 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:59:56.070 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:56.069+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:56.083 INFO:teuthology.orchestra.run.smithi042.stdout:moved 0 / 82 (0%) 2024-09-18T00:59:56.083 INFO:teuthology.orchestra.run.smithi042.stdout:avg 20.5 2024-09-18T00:59:56.083 INFO:teuthology.orchestra.run.smithi042.stdout:stddev 1.5 -> 1.5 (expected baseline 3.9211) 2024-09-18T00:59:56.083 INFO:teuthology.orchestra.run.smithi042.stdout:min osd.5 with 19 -> 19 pgs (0.926829 -> 0.926829 * mean) 2024-09-18T00:59:56.083 INFO:teuthology.orchestra.run.smithi042.stdout:max osd.4 with 23 -> 23 pgs (1.12195 -> 1.12195 * mean) 2024-09-18T00:59:56.084 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:59:56.084 INFO:teuthology.orchestra.run.smithi042.stdout:oload 1000 2024-09-18T00:59:56.084 INFO:teuthology.orchestra.run.smithi042.stdout:max_change 0.05 2024-09-18T00:59:56.084 INFO:teuthology.orchestra.run.smithi042.stdout:max_change_osds 4 2024-09-18T00:59:56.084 INFO:teuthology.orchestra.run.smithi042.stdout:average_utilization 0.0023 2024-09-18T00:59:56.084 INFO:teuthology.orchestra.run.smithi042.stdout:overload_utilization 0.0228 2024-09-18T00:59:56.084 INFO:teuthology.orchestra.run.smithi042.stderr:no change 2024-09-18T00:59:56.104 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T00:59:56.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-18T00:59:56.118 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:56.118 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:56.132 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:59:56.176 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:56.177+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:56.231 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:56.231 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:56.254 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:56.254 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:56.271 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:56.269+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:56.272 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:56.272 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:56.291 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:56.291 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:56.303 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T00:59:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T00:59:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 126.892235145, 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T00:59:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125622 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 344064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673346+0000", 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.628099978000002, 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096365904000000002, 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125622 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673347+0000", 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7599999999999997e-07 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673354+0000", 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.486e-06 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673361+0000", 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5520000000000001e-06 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673365+0000", 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7129999999999996e-06 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768923+0000", 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095557673999999995 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768994+0000", 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0847999999999997e-05 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769712+0000", 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000718055 2024-09-18T00:59:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125623 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673371+0000", 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.628075564, 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096098244999999999, 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125623 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8200000000000001e-07 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673372+0000", 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5519999999999999e-06 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673376+0000", 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.512e-06 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673379+0000", 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1580000000000001e-06 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769200+0000", 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095820946000000004 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769317+0000", 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117003 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769469+0000", 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015179200000000001 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125624 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673382+0000", 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.628063699999998, 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101719451, 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125624 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673383+0000", 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.23e-07 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673384+0000", 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3060000000000001e-06 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673387+0000", 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2650000000000001e-06 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673390+0000", 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7810000000000001e-06 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769495+0000", 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096104599999999998 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769571+0000", 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6089999999999998e-05 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775102+0000", 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055311859999999996 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125625 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673393+0000", 2024-09-18T00:59:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.628052656000001, 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102242338, 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125625 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673394+0000", 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3700000000000001e-07 2024-09-18T00:59:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673395+0000", 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2839999999999999e-06 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673398+0000", 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1379999999999999e-06 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673401+0000", 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7149999999999998e-06 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775140+0000", 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101739127 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775212+0000", 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1984999999999996e-05 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775636+0000", 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000423752 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125627 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673404+0000", 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.628041738999997, 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10248196399999999, 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125627 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673405+0000", 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0699999999999998e-07 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673406+0000", 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2130000000000001e-06 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673409+0000", 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3400000000000002e-06 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673412+0000", 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7360000000000001e-06 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775665+0000", 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102253552 2024-09-18T00:59:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775734+0000", 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8019e-05 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775886+0000", 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015279700000000001 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125628 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673415+0000", 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.628030641000002, 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.1031562, 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125628 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673416+0000", 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.91e-07 2024-09-18T00:59:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673418+0000", 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9259999999999999e-06 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673423+0000", 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6149999999999996e-06 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673428+0000", 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1460000000000004e-06 2024-09-18T00:59:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776149+0000", 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10272025899999999 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776218+0000", 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9207000000000003e-05 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776572+0000", 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035375599999999999 2024-09-18T00:59:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125629 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673435+0000", 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.628011090000001, 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103390994, 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125629 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1800000000000001e-07 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673438+0000", 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4310000000000001e-06 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673446+0000", 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0290000000000005e-06 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673451+0000", 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3630000000000002e-06 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776600+0000", 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10314894500000001 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776668+0000", 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7963e-05 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776826+0000", 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015784499999999999 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125631 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673458+0000", 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.627987840000003, 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.104002186, 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:56.340 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125631 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673459+0000", 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4499999999999997e-07 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673461+0000", 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.351e-06 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673467+0000", 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3389999999999998e-06 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673472+0000", 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9490000000000004e-06 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777062+0000", 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103589771 2024-09-18T00:59:56.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777130+0000", 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8065999999999995e-05 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777460+0000", 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000330265 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125632 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673479+0000", 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.627967052000002, 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10711156300000001, 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125632 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T00:59:56.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6100000000000002e-07 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673482+0000", 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1210000000000001e-06 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673488+0000", 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1659999999999999e-06 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673493+0000", 2024-09-18T00:59:56.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.989e-06 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777489+0000", 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10399652600000001 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777565+0000", 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6219999999999999e-05 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780591+0000", 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030251800000000001 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125633 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673500+0000", 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.627946289999997, 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10735947699999999, 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.349 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125633 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9099999999999999e-07 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673503+0000", 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4360000000000001e-06 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673509+0000", 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3969999999999999e-06 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673515+0000", 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7640000000000002e-06 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780622+0000", 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.107107533 2024-09-18T00:59:56.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780692+0000", 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9735999999999998e-05 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780859+0000", 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016721999999999999 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.353 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126363 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.711354+0000", 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.590092149, 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10567230599999999, 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126363 2024-09-18T00:59:56.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711355+0000", 2024-09-18T00:59:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711360+0000", 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9740000000000001e-06 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711380+0000", 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9769e-05 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711396+0000", 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5947000000000002e-05 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817011+0000", 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.105615185 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817026+0000", 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5430999999999998e-05 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.357 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126375 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721909+0000", 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.579537498999997, 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095654577000000005, 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.358 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126375 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721909+0000", 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721909+0000", 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721909+0000", 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3300000000000003e-07 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721912+0000", 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.683e-06 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721918+0000", 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2380000000000002e-06 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721923+0000", 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9180000000000002e-06 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817557+0000", 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095633911000000002 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817563+0000", 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3940000000000001e-06 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126376 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721930+0000", 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.579515864000001, 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097651948000000002, 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126376 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721931+0000", 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T00:59:56.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721933+0000", 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721939+0000", 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721944+0000", 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0089999999999998e-06 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817590+0000", 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095646089000000004 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.819582+0000", 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019924669999999999 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126427 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721951+0000", 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.579495330999997, 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095671891999999994, 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126427 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9499999999999998e-07 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721954+0000", 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4700000000000001e-06 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721960+0000", 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4629999999999998e-06 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721965+0000", 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0590000000000002e-06 2024-09-18T00:59:56.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817614+0000", 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095649026999999998 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817623+0000", 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4780000000000008e-06 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126428 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721973+0000", 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.579473143999998, 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095683878999999999, 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126428 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.370 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T00:59:56.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721976+0000", 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4279999999999999e-06 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721983+0000", 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7499999999999997e-06 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721988+0000", 2024-09-18T00:59:56.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4349999999999996e-06 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817644+0000", 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095656525000000006 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817657+0000", 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2312e-05 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.373 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126429 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721996+0000", 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.579450104000003, 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095687929000000005, 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126429 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9400000000000001e-07 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721999+0000", 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.649e-06 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722006+0000", 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8299999999999998e-06 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722011+0000", 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6970000000000001e-06 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817678+0000", 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095667143999999996 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817684+0000", 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2149999999999998e-06 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126430 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722018+0000", 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.579427981000002, 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095682481999999999, 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.378 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126430 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722019+0000", 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722021+0000", 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.419e-06 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722027+0000", 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0750000000000003e-06 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722033+0000", 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6030000000000004e-06 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817695+0000", 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095662607999999996 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817701+0000", 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1359999999999996e-06 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126431 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722040+0000", 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.579406077999998, 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678416000000002, 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.382 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126431 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8000000000000001e-07 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722043+0000", 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722049+0000", 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3219999999999998e-06 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722054+0000", 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817715+0000", 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095660429000000005 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817718+0000", 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7349999999999998e-06 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126432 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722061+0000", 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.579384656999999, 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095671141000000001, 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T00:59:56.386 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126432 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722061+0000", 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722061+0000", 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T00:59:56.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722062+0000", 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5400000000000002e-07 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722064+0000", 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2730000000000001e-06 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722070+0000", 2024-09-18T00:59:56.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.117e-06 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722075+0000", 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3310000000000003e-06 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817729+0000", 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095653338000000004 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T00:59:56.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817733+0000", 2024-09-18T00:59:56.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7280000000000001e-06 2024-09-18T00:59:56.390 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.390 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.390 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.390 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T00:59:56.390 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T00:59:56.390 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:56.390 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:59:56.391 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:56.392 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:56.392 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:56.393 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:56.393 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:59:56.393 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:56.389+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:56.394 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:56.394 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:56.394 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:56.492 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T00:59:56.493+0000 7f72bb180640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-18T00:59:56.492 INFO:teuthology.orchestra.run.smithi042.stdout:moved 4 / 82 (4.87805%) 2024-09-18T00:59:56.492 INFO:teuthology.orchestra.run.smithi042.stdout:avg 20.5 2024-09-18T00:59:56.493 INFO:teuthology.orchestra.run.smithi042.stdout:stddev 1.5 -> 1.11803 (expected baseline 3.9211) 2024-09-18T00:59:56.493 INFO:teuthology.orchestra.run.smithi042.stdout:min osd.5 with 19 -> 20 pgs (0.926829 -> 0.97561 * mean) 2024-09-18T00:59:56.493 INFO:teuthology.orchestra.run.smithi042.stdout:max osd.4 with 23 -> 19 pgs (1.12195 -> 0.926829 * mean) 2024-09-18T00:59:56.493 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:59:56.493 INFO:teuthology.orchestra.run.smithi042.stdout:oload 110 2024-09-18T00:59:56.493 INFO:teuthology.orchestra.run.smithi042.stdout:max_change 1 2024-09-18T00:59:56.493 INFO:teuthology.orchestra.run.smithi042.stdout:max_change_osds 4 2024-09-18T00:59:56.493 INFO:teuthology.orchestra.run.smithi042.stdout:average_utilization 0.0022 2024-09-18T00:59:56.493 INFO:teuthology.orchestra.run.smithi042.stdout:overload_utilization 0.0024 2024-09-18T00:59:56.493 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 weight 1.0000 -> 0.8891 2024-09-18T00:59:56.493 INFO:teuthology.orchestra.run.smithi042.stderr:no change 2024-09-18T00:59:56.494 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:56.489+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:56.515 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T00:59:56.515 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-18T00:59:56.567 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:56.567 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:56.581 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:59:56.610 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:56.609+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:56.693 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:56.693+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:56.702 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:56.702 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:56.716 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:59:56.794 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:56.793+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:56.832 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:56.832 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:56.848 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:59:56.895 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:56.893+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:56.918 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T00:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout:moved 4 / 82 (4.87805%) 2024-09-18T00:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout:avg 20.5 2024-09-18T00:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout:stddev 1.5 -> 1.11803 (expected baseline 3.9211) 2024-09-18T00:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout:min osd.5 with 19 -> 20 pgs (0.926829 -> 0.97561 * mean) 2024-09-18T00:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout:max osd.4 with 23 -> 19 pgs (1.12195 -> 0.926829 * mean) 2024-09-18T00:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout:oload 110 2024-09-18T00:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout:max_change 3 2024-09-18T00:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout:max_change_osds 4 2024-09-18T00:59:56.924 INFO:teuthology.orchestra.run.smithi042.stdout:average_utilization 0.0022 2024-09-18T00:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout:overload_utilization 0.0024 2024-09-18T00:59:56.925 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 weight 1.0000 -> 0.8891 2024-09-18T00:59:56.945 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-18T00:59:56.945 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-18T00:59:56.995 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:56.993+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:57.020 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:57.020 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:57.034 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:59:57.120 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:57.121+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:57.196 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:57.193+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:57.201 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:57.201 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:57.215 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:59:57.276 INFO:teuthology.orchestra.run.smithi042.stdout:moved 0 / 82 (0%) 2024-09-18T00:59:57.276 INFO:teuthology.orchestra.run.smithi042.stdout:avg 20.5 2024-09-18T00:59:57.276 INFO:teuthology.orchestra.run.smithi042.stdout:stddev 1.11803 -> 1.11803 (expected baseline 3.9211) 2024-09-18T00:59:57.276 INFO:teuthology.orchestra.run.smithi042.stdout:min osd.4 with 19 -> 19 pgs (0.926829 -> 0.926829 * mean) 2024-09-18T00:59:57.276 INFO:teuthology.orchestra.run.smithi042.stdout:max osd.6 with 22 -> 22 pgs (1.07317 -> 1.07317 * mean) 2024-09-18T00:59:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:59:57.277 INFO:teuthology.orchestra.run.smithi042.stdout:oload 1000 2024-09-18T00:59:57.277 INFO:teuthology.orchestra.run.smithi042.stdout:max_change 1 2024-09-18T00:59:57.277 INFO:teuthology.orchestra.run.smithi042.stdout:max_change_osds 4 2024-09-18T00:59:57.277 INFO:teuthology.orchestra.run.smithi042.stdout:average_utilization 0.0022 2024-09-18T00:59:57.277 INFO:teuthology.orchestra.run.smithi042.stdout:overload_utilization 0.0223 2024-09-18T00:59:57.277 INFO:teuthology.orchestra.run.smithi042.stderr:no change 2024-09-18T00:59:57.293 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-18T00:59:57.294 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-18T00:59:57.338 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:57.337+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:57.382 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:57.382 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:57.396 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:59:57.398 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:57.397+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:57.499 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:57.497+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:57.526 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:57.526 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:57.541 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:59:57.600 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:57.597+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:57.642 INFO:teuthology.orchestra.run.smithi042.stdout:moved 0 / 82 (0%) 2024-09-18T00:59:57.642 INFO:teuthology.orchestra.run.smithi042.stdout:avg 20.5 2024-09-18T00:59:57.642 INFO:teuthology.orchestra.run.smithi042.stdout:stddev 1.11803 -> 1.11803 (expected baseline 3.9211) 2024-09-18T00:59:57.642 INFO:teuthology.orchestra.run.smithi042.stdout:min osd.4 with 19 -> 19 pgs (0.926829 -> 0.926829 * mean) 2024-09-18T00:59:57.642 INFO:teuthology.orchestra.run.smithi042.stdout:max osd.6 with 22 -> 22 pgs (1.07317 -> 1.07317 * mean) 2024-09-18T00:59:57.642 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T00:59:57.642 INFO:teuthology.orchestra.run.smithi042.stdout:oload 1000 2024-09-18T00:59:57.642 INFO:teuthology.orchestra.run.smithi042.stdout:max_change 1 2024-09-18T00:59:57.642 INFO:teuthology.orchestra.run.smithi042.stdout:max_change_osds 4 2024-09-18T00:59:57.642 INFO:teuthology.orchestra.run.smithi042.stdout:average_utilization 0.0022 2024-09-18T00:59:57.643 INFO:teuthology.orchestra.run.smithi042.stdout:overload_utilization 0.0223 2024-09-18T00:59:57.643 INFO:teuthology.orchestra.run.smithi042.stderr:no change 2024-09-18T00:59:57.660 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:57.660 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:57.675 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:59:57.718 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:57.717+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:57.801 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:57.797+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:57.836 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:57.837 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:57.851 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:59:57.942 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:57.941+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:58.004 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:58.001+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:58.023 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:58.023 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:58.039 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:59:58.105 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:58.105+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:58.206 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T00:59:58.205+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T00:59:58.210 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:58.210 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:58.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:59:58.310 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T00:59:58.309+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T00:59:58.391 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:58.391 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:58.404 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:59:58.406 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:59:58.407 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:58.405+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:58.508 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T00:59:58.505+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T00:59:58.554 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T00:59:58.555 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:59:58.572 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T00:59:58.572 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:59:58.578 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T00:59:58.578 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:59:58.590 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T00:59:58.590 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:59:58.593 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:59:58.607 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T00:59:58.608 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:59:58.608 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:58.609+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:58.629 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T00:59:58.629 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:59:58.650 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T00:59:58.650 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:59:58.673 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T00:59:58.673 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:59:58.693 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T00:59:58.694 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:59:58.709 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:58.709+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:58.710 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:58.710 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:58.710 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:58.710 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:58.710 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:58.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:59:58.810 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:58.809+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T00:59:58.885 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.80590441999999, 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T00:59:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T00:59:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.804974829, 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T00:59:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.80496076899999, 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T00:59:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.80367019400001, 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T00:59:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.80364972199999, 2024-09-18T00:59:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T00:59:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T00:59:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T00:59:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.80085526900001, 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T00:59:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T00:59:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.800840085, 2024-09-18T00:59:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T00:59:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.800817139, 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T00:59:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T00:59:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.7966395, 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T00:59:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.79661828299999, 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T00:59:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T00:59:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.7965969, 2024-09-18T00:59:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T00:59:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.792697046, 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T00:59:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.792682845, 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T00:59:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T00:59:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.79266480800001, 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T00:59:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.79265074200001, 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T00:59:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.78862539100001, 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.788611379, 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T00:59:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.788597415, 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T00:59:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T00:59:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.02673946100001, 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T00:59:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T00:59:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 111.90126309, 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T00:59:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T00:59:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T00:59:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:59:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:58.979 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:58.979 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:59:58.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:59:58.981 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:58.977+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:59.081 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:59.077+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:59.183 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:59.183 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:59.183 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T00:59:59.183 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:59.183 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:59.181+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:59.203 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:59:59.290 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:59.289+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:59.371 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:59.371 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T00:59:59.371 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T00:59:59.371 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T00:59:59.371 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:59.385 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:59.381+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:59.386 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:59:59.486 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T00:59:59.485+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T00:59:59.553 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T00:59:59.553 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T00:59:59.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T00:59:59.553 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T00:59:59.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.553 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98052 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4179968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.553 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:59.553 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.109638668000002, 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131360583, 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98052 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440037+0000", 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440040+0000", 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7159999999999999e-06 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440048+0000", 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7169999999999997e-06 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440056+0000", 2024-09-18T00:59:59.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1860000000000003e-06 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.570989+0000", 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13093384999999999 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571175+0000", 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018504799999999999 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571397+0000", 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022245899999999999 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98053 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4466688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.109610798999995, 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13182195299999999, 2024-09-18T00:59:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98053 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440065+0000", 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440068+0000", 2024-09-18T00:59:59.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6699999999999998e-06 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440076+0000", 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7069999999999998e-06 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440083+0000", 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9929999999999998e-06 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571475+0000", 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13139158100000001 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571657+0000", 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018247099999999999 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571886+0000", 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022892400000000001 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98054 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2113536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.109582939000006, 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13228911600000001, 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98054 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440093+0000", 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440096+0000", 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440105+0000", 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7739999999999993e-06 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440112+0000", 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2130000000000002e-06 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571970+0000", 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13185812199999999 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572159+0000", 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000188665 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572381+0000", 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022273200000000001 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.567 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98055 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2400256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.109552758999996, 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273966600000001, 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98055 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440123+0000", 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-18T00:59:59.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440126+0000", 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440137+0000", 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0543000000000001e-05 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440144+0000", 2024-09-18T00:59:59.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572457+0000", 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132312807 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572636+0000", 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882299999999999 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572862+0000", 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022645200000000001 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98056 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2686976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.109522182000006, 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13308801200000001, 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98056 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440154+0000", 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5300000000000004e-07 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440156+0000", 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.903e-06 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440165+0000", 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7600000000000008e-06 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440174+0000", 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6969999999999999e-06 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572960+0000", 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132785984 2024-09-18T00:59:59.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573087+0000", 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012728699999999999 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573241+0000", 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153728 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98057 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2973696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.109492588999998, 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13357376900000001, 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98057 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440183+0000", 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440186+0000", 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8700000000000001e-06 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440195+0000", 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7099999999999996e-06 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440202+0000", 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2429999999999999e-06 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573338+0000", 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13313578400000001 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573526+0000", 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018780399999999999 2024-09-18T00:59:59.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573756+0000", 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230755 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98058 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.109464705999997, 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13387749400000001, 2024-09-18T00:59:59.581 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98058 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440627+0000", 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041660699999999999 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440634+0000", 2024-09-18T00:59:59.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.596e-06 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440650+0000", 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6164e-05 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440660+0000", 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.023e-05 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573833+0000", 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13317317000000001 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573945+0000", 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111731 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574088+0000", 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000142996 2024-09-18T00:59:59.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98059 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3547136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.108997359, 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13391484000000001, 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.586 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98059 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441760+0000", 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001082185 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441780+0000", 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0463e-05 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441829+0000", 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8763000000000001e-05 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441849+0000", 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9423e-05 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574173+0000", 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13232460400000001 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574367+0000", 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019403000000000001 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574593+0000", 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000225372 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.590 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98060 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3833856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.107795277999998, 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13320725899999999, 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98060 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441881+0000", 2024-09-18T00:59:59.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6160000000000001e-06 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441889+0000", 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8439999999999998e-06 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441909+0000", 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9856000000000001e-05 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441925+0000", 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6039999999999999e-05 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574669+0000", 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13274412799999999 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574860+0000", 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019038599999999999 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575087+0000", 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022738899999999999 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98061 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4120576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.107728522000002, 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13362664099999999, 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98061 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441948+0000", 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441954+0000", 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3099999999999997e-06 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441981+0000", 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7095999999999999e-05 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441998+0000", 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6237000000000001e-05 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575165+0000", 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13316721200000001 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575355+0000", 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000190218 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575573+0000", 2024-09-18T00:59:59.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021824799999999999 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98062 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4407296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.107656552999998, 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13404179099999999, 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98062 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445789+0000", 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037705569999999999 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445805+0000", 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.558e-05 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445846+0000", 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445871+0000", 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5072999999999999e-05 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575651+0000", 2024-09-18T00:59:59.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12977999600000001 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575835+0000", 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000183931 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576060+0000", 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022558800000000001 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98064 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2115584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.103692113999998, 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13103894799999999, 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98064 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.605 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445985+0000", 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445991+0000", 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446014+0000", 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3056000000000001e-05 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446030+0000", 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5662999999999999e-05 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576622+0000", 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13059273900000001 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576805+0000", 2024-09-18T00:59:59.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018304899999999999 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577022+0000", 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216481 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98065 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2402304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.103624998000001, 2024-09-18T00:59:59.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131457463, 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98065 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446051+0000", 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1260000000000001e-06 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446057+0000", 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2040000000000002e-06 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446077+0000", 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446092+0000", 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5124e-05 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577098+0000", 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13100595100000001 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577284+0000", 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000186152 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577508+0000", 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022314000000000001 2024-09-18T00:59:59.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98066 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2689024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.103562792000005, 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13189522100000001, 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98066 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446113+0000", 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0240000000000001e-06 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446120+0000", 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446137+0000", 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7479e-05 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446152+0000", 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4633e-05 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577587+0000", 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13143572100000001 2024-09-18T00:59:59.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577786+0000", 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019872999999999999 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578008+0000", 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022141399999999999 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98067 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2975744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.103505128999998, 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132319567, 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98067 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.619 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446171+0000", 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446178+0000", 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699999999999997e-06 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446196+0000", 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7989999999999999e-05 2024-09-18T00:59:59.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446213+0000", 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7493999999999999e-05 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578083+0000", 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13186995000000001 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578266+0000", 2024-09-18T00:59:59.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018321099999999999 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578490+0000", 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022316600000000001 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98068 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3262464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:59.623 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.103444222999997, 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273661, 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98068 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446232+0000", 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.187e-06 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446238+0000", 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446257+0000", 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8732999999999998e-05 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446272+0000", 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5532999999999998e-05 2024-09-18T00:59:59.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578569+0000", 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132296631 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578749+0000", 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017978300000000001 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578967+0000", 2024-09-18T00:59:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218736 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98069 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3549184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.103384550000001, 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13311771, 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98069 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446292+0000", 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446298+0000", 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4470000000000003e-06 2024-09-18T00:59:59.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446340+0000", 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1705999999999997e-05 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446358+0000", 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7873999999999999e-05 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579072+0000", 2024-09-18T00:59:59.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13271396399999999 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579213+0000", 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014133900000000001 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579408+0000", 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019492700000000001 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.632 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98070 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3835904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.103291107000004, 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133517145, 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.633 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98070 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446385+0000", 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446393+0000", 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2629999999999997e-06 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446413+0000", 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9930000000000001e-05 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446428+0000", 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5336e-05 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579494+0000", 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13306637399999999 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579682+0000", 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018740900000000001 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579901+0000", 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021946300000000001 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.637 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98071 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.103227941, 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133931615, 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98071 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449904+0000", 2024-09-18T00:59:59.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003456822 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449945+0000", 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0896000000000003e-05 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449971+0000", 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5996999999999999e-05 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449986+0000", 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.49e-05 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579978+0000", 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.129992723 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580157+0000", 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179006 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580379+0000", 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022127099999999999 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98073 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4696064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.099618813999996, 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13113503100000001, 2024-09-18T00:59:59.642 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98073 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450057+0000", 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450060+0000", 2024-09-18T00:59:59.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5329999999999999e-06 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450072+0000", 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1919999999999999e-05 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450080+0000", 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9500000000000001e-06 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580951+0000", 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.130871128 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581057+0000", 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010501399999999999 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581191+0000", 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013479900000000001 2024-09-18T00:59:59.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.647 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T00:59:59.647 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T00:59:59.647 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T00:59:59.648 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:59:59.649 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T00:59:59.649+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T00:59:59.749 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T00:59:59.749+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T00:59:59.778 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:59.779 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:59.779 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T00:59:59.779 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:59.792 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:59:59.851 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T00:59:59.849+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T00:59:59.917 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T00:59:59.917 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T00:59:59.917 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T00:59:59.917 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T00:59:59.917 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T00:59:59.931 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:59:59.940 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T00:59:59.976 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T00:59:59.977+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:00.053 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:00.049+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:00.065 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:00.065 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:00.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:00.065 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:00.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.065 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:00.065 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:00.065 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 131.28707116800001, 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T01:00:00.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:00.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.071 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87406 3.19 3:9d734bda:::rbd_data.117c8934add3.000000000000007c:head [stat,delete] snapc 1f=[1f] RETRY=8 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 71.639148046000003, 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097024104999999999, 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87406 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425532+0000", 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027458309999999998 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425537+0000", 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2839999999999997e-06 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425551+0000", 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3682999999999999e-05 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425559+0000", 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5329999999999997e-06 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515035+0000", 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.089476411000000006 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515168+0000", 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013309699999999999 2024-09-18T01:00:00.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515475+0000", 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030662199999999999 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519553+0000", 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040781259999999996 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:00.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519665+0000", 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112669 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519748+0000", 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2270999999999999e-05 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519810+0000", 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2577999999999996e-05 2024-09-18T01:00:00.077 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87414 3.19 3:9c145b25:::rbd_data.117c8934add3.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 71.636324506999998, 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095886693999999995, 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.078 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87414 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425618+0000", 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425625+0000", 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0870000000000001e-06 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425630+0000", 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.262e-06 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518417+0000", 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092787816999999995 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518593+0000", 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017569100000000001 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518939+0000", 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034572299999999998 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521320+0000", 2024-09-18T01:00:00.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023809619999999999 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521343+0000", 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3317000000000001e-05 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521422+0000", 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480999999999996e-05 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521496+0000", 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4909999999999999e-05 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99159 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 393216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.692668157, 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096008205999999999, 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.084 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99159 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369409+0000", 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014342599999999999 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369415+0000", 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.31e-06 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369426+0000", 2024-09-18T01:00:00.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1677e-05 2024-09-18T01:00:00.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369435+0000", 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.464949+0000", 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095513718999999997 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465099+0000", 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014984300000000001 2024-09-18T01:00:00.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465274+0000", 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017554499999999999 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99160 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2336768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.692483588000002, 2024-09-18T01:00:00.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096064584999999994, 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99160 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:00.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369564+0000", 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113309 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369569+0000", 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5269999999999998e-06 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369582+0000", 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2952999999999999e-05 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369592+0000", 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0939999999999997e-06 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465328+0000", 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095736285000000004 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465403+0000", 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5498000000000001e-05 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465515+0000", 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111919 2024-09-18T01:00:00.093 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99161 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 413696~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.692325959000001, 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096302270999999995, 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.094 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99161 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:00.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373792+0000", 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004184063 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373801+0000", 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3729999999999993e-06 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373818+0000", 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7280000000000001e-05 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373829+0000", 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0540000000000001e-05 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465596+0000", 2024-09-18T01:00:00.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.091767910999999994 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465740+0000", 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143683 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465911+0000", 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000170421 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99172 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2459648~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.687730616000003, 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095842035000000006, 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.099 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99172 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9299999999999997e-07 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374207+0000", 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8899999999999999e-06 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374216+0000", 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3669999999999997e-06 2024-09-18T01:00:00.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374223+0000", 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9759999999999998e-06 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469763+0000", 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095540307000000005 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469894+0000", 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013135600000000001 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470046+0000", 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015144600000000001 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99173 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2480128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.687703055999997, 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096195265000000002, 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99173 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374232+0000", 2024-09-18T01:00:00.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4600000000000004e-07 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374235+0000", 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8569999999999999e-06 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374243+0000", 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5930000000000006e-06 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374250+0000", 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9330000000000004e-06 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470113+0000", 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095862952000000001 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470259+0000", 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014534300000000001 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470426+0000", 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016784100000000001 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99174 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2500608~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.687674749999999, 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096540567999999993, 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.108 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99174 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374260+0000", 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2600000000000002e-07 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374263+0000", 2024-09-18T01:00:00.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8770000000000002e-06 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374272+0000", 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6070000000000008e-06 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374279+0000", 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8430000000000004e-06 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470492+0000", 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096213513000000001 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470632+0000", 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013954899999999999 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470800+0000", 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016845299999999999 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.112 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99175 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 536576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.687646854, 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096867154999999996, 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.113 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99175 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374288+0000", 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9999999999999997e-07 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374291+0000", 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.92e-06 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374300+0000", 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5939999999999994e-06 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374307+0000", 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470858+0000", 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096551085999999994 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470990+0000", 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013223900000000001 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471155+0000", 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016463299999999999 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.117 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99176 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2521088~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.687618903999997, 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097204272999999994, 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99176 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374316+0000", 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1999999999999999e-07 2024-09-18T01:00:00.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374336+0000", 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9879999999999999e-05 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374346+0000", 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0353999999999999e-05 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374354+0000", 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7130000000000005e-06 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471210+0000", 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096856271999999993 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471350+0000", 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013940900000000001 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471520+0000", 2024-09-18T01:00:00.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000169925 2024-09-18T01:00:00.122 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.122 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.122 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.122 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99177 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 557056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.122 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:00.122 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.687570041000001, 2024-09-18T01:00:00.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097507736999999997, 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99177 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:00.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377919+0000", 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035553070000000002 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377927+0000", 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4569999999999999e-06 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377943+0000", 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5563e-05 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377953+0000", 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0413e-05 2024-09-18T01:00:00.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471578+0000", 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093625036999999994 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471710+0000", 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013170900000000001 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471872+0000", 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162251 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99183 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 618496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.683782004999998, 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095979540000000002, 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99183 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378153+0000", 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.23e-07 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378156+0000", 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9340000000000002e-06 2024-09-18T01:00:00.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378164+0000", 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4630000000000001e-06 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378173+0000", 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8130000000000002e-06 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473851+0000", 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678257000000003 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473982+0000", 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000130875 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474132+0000", 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014947499999999999 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.131 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99184 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2603008~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.683725463000002, 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096289871999999999, 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.132 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99184 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378210+0000", 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2999999999999999e-07 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378213+0000", 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378223+0000", 2024-09-18T01:00:00.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378230+0000", 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5830000000000001e-06 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474192+0000", 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961742000000003 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474324+0000", 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000132099 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474499+0000", 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174551 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99185 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 638976~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.683691035999999, 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096618187999999994, 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99185 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.137 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378244+0000", 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378248+0000", 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5130000000000001e-06 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378256+0000", 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8429999999999999e-06 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378263+0000", 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9870000000000002e-06 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474554+0000", 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096290693999999996 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474692+0000", 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013761900000000001 2024-09-18T01:00:00.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474861+0000", 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016967499999999999 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99186 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 659456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.683661485999998, 2024-09-18T01:00:00.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096958847000000001, 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99186 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:00.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1699999999999997e-07 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378277+0000", 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0800000000000002e-06 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378285+0000", 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5699999999999993e-06 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378292+0000", 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9600000000000003e-06 2024-09-18T01:00:00.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474924+0000", 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096632207999999997 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475067+0000", 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143175 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475232+0000", 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016413699999999999 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99187 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 679936~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.683633213, 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097293710000000005, 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.146 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99187 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378302+0000", 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378305+0000", 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.066e-06 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378315+0000", 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0443e-05 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378323+0000", 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3239999999999997e-06 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475291+0000", 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096968726000000005 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475430+0000", 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013847499999999999 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475595+0000", 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016497899999999999 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99188 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 700416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.150 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.683602377, 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097626477000000003, 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99188 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:00.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:00.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381945+0000", 2024-09-18T01:00:00.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036127569999999999 2024-09-18T01:00:00.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381960+0000", 2024-09-18T01:00:00.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.554e-05 2024-09-18T01:00:00.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381977+0000", 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6733000000000001e-05 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381987+0000", 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.031e-05 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475651+0000", 2024-09-18T01:00:00.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093664175000000002 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475789+0000", 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000137181 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475958+0000", 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016978100000000001 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99195 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 761856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.679729571999999, 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961833999999996, 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.156 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99195 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382208+0000", 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0469999999999998e-06 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382217+0000", 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5159999999999994e-06 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382224+0000", 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.64e-06 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.477889+0000", 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095664955999999995 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478012+0000", 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012299899999999999 2024-09-18T01:00:00.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478166+0000", 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015497899999999999 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99196 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 88064~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.679701641999998, 2024-09-18T01:00:00.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096312801000000003, 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99196 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:00.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385636+0000", 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034037450000000001 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385642+0000", 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9900000000000002e-06 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385659+0000", 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6753e-05 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385669+0000", 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9170000000000001e-06 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478225+0000", 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092555514000000005 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478366+0000", 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014198699999999999 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478545+0000", 2024-09-18T01:00:00.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017889499999999999 2024-09-18T01:00:00.165 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.165 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.165 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.165 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.165 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.165 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:00.166 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:00:00.167 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:00.165+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:00.267 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:00.265+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:00.337 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:00.337 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:00.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.337 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160401 3.0 3.baf36160 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-18T01:00:00.337 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:00.337 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.26630621799999998, 2024-09-18T01:00:00.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.26638343799999997, 2024-09-18T01:00:00.337 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.337 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "waiting for peered", 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160401 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071217+0000", 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4924e-05 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071229+0000", 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.166e-05 2024-09-18T01:00:00.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071272+0000", 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2506000000000003e-05 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071294+0000", 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2789000000000002e-05 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071664+0000", 2024-09-18T01:00:00.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036966100000000002 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071694+0000", 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0207000000000001e-05 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160404 3.0 3.9ebb3120 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.266129165, 2024-09-18T01:00:00.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.26633187000000003, 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "waiting for peered", 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160404 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071381+0000", 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7829999999999999e-06 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071390+0000", 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.02e-06 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071417+0000", 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6798999999999999e-05 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071434+0000", 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6983e-05 2024-09-18T01:00:00.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071769+0000", 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033536500000000001 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071788+0000", 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.345 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.346 INFO:teuthology.orchestra.run.smithi195.stdout: ], 2024-09-18T01:00:00.346 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 2 2024-09-18T01:00:00.346 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:00.352 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:00:00.369 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:00.365+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:00.469 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:00.469+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:00.521 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:00.521 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:00.521 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:00.521 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:00.521 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:00.535 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:00:00.570 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:00.569+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:00.671 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:00.669+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:00.701 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 131.290389933, 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T01:00:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125622 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 344064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.026254766000001, 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096365904000000002, 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125622 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673347+0000", 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7599999999999997e-07 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673354+0000", 2024-09-18T01:00:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.486e-06 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673361+0000", 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5520000000000001e-06 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673365+0000", 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7129999999999996e-06 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768923+0000", 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095557673999999995 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768994+0000", 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0847999999999997e-05 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769712+0000", 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000718055 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125623 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.026230351999999, 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096098244999999999, 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125623 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8200000000000001e-07 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673372+0000", 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5519999999999999e-06 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673376+0000", 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.512e-06 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673379+0000", 2024-09-18T01:00:00.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1580000000000001e-06 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769200+0000", 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095820946000000004 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769317+0000", 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117003 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769469+0000", 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015179200000000001 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125624 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.026218487999998, 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101719451, 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.717 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125624 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:00.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673383+0000", 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.23e-07 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673384+0000", 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3060000000000001e-06 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673387+0000", 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673390+0000", 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769495+0000", 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096104599999999998 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769571+0000", 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6089999999999998e-05 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775102+0000", 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055311859999999996 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125625 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.026207444000001, 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102242338, 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125625 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.722 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673394+0000", 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673395+0000", 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2839999999999999e-06 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673398+0000", 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1379999999999999e-06 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673401+0000", 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7149999999999998e-06 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775140+0000", 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101739127 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775212+0000", 2024-09-18T01:00:00.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1984999999999996e-05 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775636+0000", 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000423752 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125627 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:00.726 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.026196527000003, 2024-09-18T01:00:00.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10248196399999999, 2024-09-18T01:00:00.727 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.727 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.727 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.727 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.727 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.727 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.727 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125627 2024-09-18T01:00:00.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.727 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673405+0000", 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0699999999999998e-07 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673406+0000", 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2130000000000001e-06 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673409+0000", 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3400000000000002e-06 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673412+0000", 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7360000000000001e-06 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775665+0000", 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102253552 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775734+0000", 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8019e-05 2024-09-18T01:00:00.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775886+0000", 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015279700000000001 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125628 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.026185429000002, 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.1031562, 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.731 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125628 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673416+0000", 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.91e-07 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673418+0000", 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:00:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673423+0000", 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6149999999999996e-06 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673428+0000", 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1460000000000004e-06 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776149+0000", 2024-09-18T01:00:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10272025899999999 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776218+0000", 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9207000000000003e-05 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776572+0000", 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035375599999999999 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125629 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.026165878, 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103390994, 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125629 2024-09-18T01:00:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673438+0000", 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4310000000000001e-06 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673446+0000", 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0290000000000005e-06 2024-09-18T01:00:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673451+0000", 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3630000000000002e-06 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776600+0000", 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10314894500000001 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776668+0000", 2024-09-18T01:00:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7963e-05 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776826+0000", 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015784499999999999 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125631 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.026142628000002, 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.104002186, 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125631 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673459+0000", 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:00:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673461+0000", 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.351e-06 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673467+0000", 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3389999999999998e-06 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673472+0000", 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777062+0000", 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103589771 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777130+0000", 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8065999999999995e-05 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777460+0000", 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000330265 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125632 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.026121840000002, 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10711156300000001, 2024-09-18T01:00:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125632 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6100000000000002e-07 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673482+0000", 2024-09-18T01:00:00.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1210000000000001e-06 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673488+0000", 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1659999999999999e-06 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673493+0000", 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.989e-06 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777489+0000", 2024-09-18T01:00:00.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10399652600000001 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777565+0000", 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6219999999999999e-05 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780591+0000", 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030251800000000001 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125633 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.026101078000003, 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10735947699999999, 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125633 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673503+0000", 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673509+0000", 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3969999999999999e-06 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673515+0000", 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7640000000000002e-06 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780622+0000", 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.107107533 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780692+0000", 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9735999999999998e-05 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780859+0000", 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016721999999999999 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126363 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.988246937, 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10567230599999999, 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.754 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126363 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711355+0000", 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9999999999999995e-07 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711360+0000", 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:00:00.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711380+0000", 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9769e-05 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711396+0000", 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5947000000000002e-05 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817011+0000", 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.105615185 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817026+0000", 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5430999999999998e-05 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.758 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126375 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721909+0000", 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.977692286999996, 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095654577000000005, 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126375 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721909+0000", 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721909+0000", 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721909+0000", 2024-09-18T01:00:00.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3300000000000003e-07 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721912+0000", 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.683e-06 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721918+0000", 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2380000000000002e-06 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721923+0000", 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9180000000000002e-06 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817557+0000", 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095633911000000002 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817563+0000", 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3940000000000001e-06 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126376 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.977670652, 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097651948000000002, 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126376 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721931+0000", 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T01:00:00.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721933+0000", 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721939+0000", 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721944+0000", 2024-09-18T01:00:00.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0089999999999998e-06 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817590+0000", 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095646089000000004 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.819582+0000", 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019924669999999999 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126427 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.977650119000003, 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095671891999999994, 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126427 2024-09-18T01:00:00.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721954+0000", 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4700000000000001e-06 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721960+0000", 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4629999999999998e-06 2024-09-18T01:00:00.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721965+0000", 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817614+0000", 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095649026999999998 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817623+0000", 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4780000000000008e-06 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126428 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.977627931999997, 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095683878999999999, 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:00.771 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126428 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:00.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721976+0000", 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4279999999999999e-06 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721983+0000", 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7499999999999997e-06 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721988+0000", 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4349999999999996e-06 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817644+0000", 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095656525000000006 2024-09-18T01:00:00.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817657+0000", 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2312e-05 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126429 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.977604892000002, 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095687929000000005, 2024-09-18T01:00:00.775 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126429 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721999+0000", 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.649e-06 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722006+0000", 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8299999999999998e-06 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722011+0000", 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6970000000000001e-06 2024-09-18T01:00:00.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817678+0000", 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095667143999999996 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817684+0000", 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2149999999999998e-06 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126430 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.977582769000001, 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095682481999999999, 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126430 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.780 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722019+0000", 2024-09-18T01:00:00.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722021+0000", 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.419e-06 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722027+0000", 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0750000000000003e-06 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722033+0000", 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817695+0000", 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095662607999999996 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817701+0000", 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1359999999999996e-06 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.783 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126431 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.977560865999997, 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678416000000002, 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.784 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126431 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722043+0000", 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722049+0000", 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3219999999999998e-06 2024-09-18T01:00:00.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722054+0000", 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817715+0000", 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095660429000000005 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817718+0000", 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126432 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722061+0000", 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.977539444999998, 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095671141000000001, 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:00.788 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126432 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722061+0000", 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722061+0000", 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722062+0000", 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5400000000000002e-07 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722064+0000", 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2730000000000001e-06 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722070+0000", 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.117e-06 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722075+0000", 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3310000000000003e-06 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817729+0000", 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095653338000000004 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:00.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817733+0000", 2024-09-18T01:00:00.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7280000000000001e-06 2024-09-18T01:00:00.792 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.792 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.792 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.792 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.792 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.792 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:00.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:00:00.794 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:00:00.795 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:00.793+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:00.895 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:00.893+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:00.957 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:00.958 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:00.974 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:00.975 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:00.981 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160424 3.5 3:a7f05bff:::rbd_header.117cf66ad0cc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.979531+0000", 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.0032379489999999999, 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032648600000000001, 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160424 2024-09-18T01:00:00.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.979531+0000", 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.979531+0000", 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.979532+0000", 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7499999999999998e-07 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.979537+0000", 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4059999999999998e-06 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.979561+0000", 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4034000000000001e-05 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.979571+0000", 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0434e-05 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.979926+0000", 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035473200000000002 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.980121+0000", 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000195355 2024-09-18T01:00:00.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.980410+0000", 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028916400000000001 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.982141+0000", 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001730802 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.982611+0000", 2024-09-18T01:00:00.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00046950099999999999 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.982765+0000", 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015422499999999999 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(client.4476.0:160424 3.5 e143/89 ondisk, result = 0)", 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.982437+0000", 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.000331856, 2024-09-18T01:00:00.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000462867, 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.982437+0000", 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.982437+0000", 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.982439+0000", 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.327e-06 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.982444+0000", 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3519999999999997e-06 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.982458+0000", 2024-09-18T01:00:00.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4022999999999999e-05 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.982466+0000", 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0199999999999994e-06 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.982573+0000", 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106643 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:00.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:00.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.982595+0000", 2024-09-18T01:00:00.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2121999999999999e-05 2024-09-18T01:00:00.991 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.991 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:00.991 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.991 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:00.991 INFO:teuthology.orchestra.run.smithi195.stdout: ], 2024-09-18T01:00:00.991 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 2 2024-09-18T01:00:00.991 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:00.992 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:00.992 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:00.995 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:00.993+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:00.998 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:00:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:01.008 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:01.032 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:01.032 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:01.052 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:01.053 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:01.071 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:01.089 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:01.089 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:01.096 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:01.097+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:01.159 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:01.160 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:01.173 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:00:01.216 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:01.217+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:01.298 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:01.297+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:01.336 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:01.336 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:01.349 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:00:01.398 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:01.397+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:01.499 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:01.497+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:01.519 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:01.519 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:01.533 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:00:01.620 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:01.621+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:01.700 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:01.700 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:01.700 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:01.697+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:01.714 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:00:01.801 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:01.801+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:01.873 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:01.873 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:01.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:00:01.936 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:01.933+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:01.970 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:00:02.018 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:02.017+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:02.038 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:02.038 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:02.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:00:02.142 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:02.141+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:02.205 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:02.201+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:02.220 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:02.220 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:02.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:00:02.306 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:02.305+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:02.405 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:02.406 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:02.405+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:02.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:00:02.507 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:02.505+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:02.587 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:02.588 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:02.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:00:02.646 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:02.645+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:02.663 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6] out_osds: [2, 3, 0, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:00:02.663 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:00:02.663 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-18T01:00:02.663 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-18T01:00:02.738 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:02.737+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:02.775 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:02.775 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:02.790 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:00:02.809 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:02.809+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:02.909 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:02.905+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:02.938 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T01:00:02.960 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:02.960 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:02.976 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:00:03.011 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-18T01:00:03.009+0000 7fdb35eeb640 -1 mon.a@0(leader).osd e145 definitely_dead 0 2024-09-18T01:00:03.011 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:03.009+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:03.108 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:00:03.111 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:03.109+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:03.135 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:03.136 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:03.152 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:00:03.212 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:03.213+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:03.264 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:03.265 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:03.283 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:03.300 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:03.313 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:03.313+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:03.317 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:03.317 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:03.323 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:03.323 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:03.323 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:03.323 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:03.323 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:03.338 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:00:03.338 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:03.339 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:03.359 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:03.359 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:03.380 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:03.381 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:03.401 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:03.401 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:03.414 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:03.413+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.37800057800001, 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.457 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T01:00:03.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T01:00:03.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T01:00:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.377070987, 2024-09-18T01:00:03.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T01:00:03.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.37705692700001, 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T01:00:03.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.375766352, 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.474 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.37574588000001, 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.478 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.372951427, 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T01:00:03.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:03.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T01:00:03.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T01:00:03.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.372936243, 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.483 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T01:00:03.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T01:00:03.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.372913297, 2024-09-18T01:00:03.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T01:00:03.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.36873565799999, 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T01:00:03.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T01:00:03.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.36871444100001, 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.36869305799999, 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.500 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T01:00:03.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.504 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.36479320399999, 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T01:00:03.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T01:00:03.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.364779003, 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T01:00:03.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.36476096600001, 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T01:00:03.513 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.517 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.3647469, 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T01:00:03.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.360721549, 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T01:00:03.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.360707537, 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T01:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T01:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T01:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T01:00:03.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.36069357299999, 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T01:00:03.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T01:00:03.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.598835619, 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.47335924799999, 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.540 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T01:00:03.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T01:00:03.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T01:00:03.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:03.545 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:03.547 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:00:03.547 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:03.545+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:03.648 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:03.645+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:03.722 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:03.722 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:03.722 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:03.722 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:03.736 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:00:03.748 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:03.749+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:03.849 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:03.845+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:03.903 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:03.903 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:03.903 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:03.903 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:03.903 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:03.917 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:00:03.955 INFO:teuthology.orchestra.run.smithi042.stderr:marked in osd.7. 2024-09-18T01:00:03.955 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:03.953+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:03.978 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-18T01:00:04.050 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:04.049+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:04.080 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:04.080 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:04.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:04.080 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:04.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.080 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98052 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4179968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.080 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440036+0000", 2024-09-18T01:00:04.080 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.636707166999997, 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131360583, 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98052 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T01:00:04.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440037+0000", 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440040+0000", 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7159999999999999e-06 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440048+0000", 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7169999999999997e-06 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440056+0000", 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1860000000000003e-06 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.570989+0000", 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13093384999999999 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571175+0000", 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018504799999999999 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571397+0000", 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022245899999999999 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.084 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98053 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4466688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440064+0000", 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.636679298000004, 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13182195299999999, 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98053 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440065+0000", 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440068+0000", 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6699999999999998e-06 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440076+0000", 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7069999999999998e-06 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440083+0000", 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9929999999999998e-06 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571475+0000", 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13139158100000001 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571657+0000", 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018247099999999999 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571886+0000", 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022892400000000001 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98054 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2113536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440092+0000", 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.636651438000001, 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13228911600000001, 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98054 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440093+0000", 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440096+0000", 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440105+0000", 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7739999999999993e-06 2024-09-18T01:00:04.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440112+0000", 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2130000000000002e-06 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571970+0000", 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13185812199999999 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572159+0000", 2024-09-18T01:00:04.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000188665 2024-09-18T01:00:04.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572381+0000", 2024-09-18T01:00:04.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022273200000000001 2024-09-18T01:00:04.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.093 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.093 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98055 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2400256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440122+0000", 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.636621258000005, 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273966600000001, 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98055 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440123+0000", 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440126+0000", 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440137+0000", 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0543000000000001e-05 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440144+0000", 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572457+0000", 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132312807 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572636+0000", 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882299999999999 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572862+0000", 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022645200000000001 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.097 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98056 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2686976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440153+0000", 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.636590681000001, 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13308801200000001, 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98056 2024-09-18T01:00:04.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440154+0000", 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5300000000000004e-07 2024-09-18T01:00:04.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440156+0000", 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.903e-06 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440165+0000", 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7600000000000008e-06 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440174+0000", 2024-09-18T01:00:04.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6969999999999999e-06 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572960+0000", 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132785984 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573087+0000", 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012728699999999999 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573241+0000", 2024-09-18T01:00:04.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153728 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98057 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2973696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440182+0000", 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.636561087999993, 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13357376900000001, 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98057 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T01:00:04.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440183+0000", 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440186+0000", 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8700000000000001e-06 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440195+0000", 2024-09-18T01:00:04.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7099999999999996e-06 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440202+0000", 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2429999999999999e-06 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573338+0000", 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13313578400000001 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573526+0000", 2024-09-18T01:00:04.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018780399999999999 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573756+0000", 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230755 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98058 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440210+0000", 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.636533205000006, 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13387749400000001, 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98058 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440627+0000", 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041660699999999999 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440634+0000", 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.596e-06 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440650+0000", 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6164e-05 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440660+0000", 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.023e-05 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573833+0000", 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13317317000000001 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573945+0000", 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111731 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574088+0000", 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000142996 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98059 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3547136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440678+0000", 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.636065857999995, 2024-09-18T01:00:04.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13391484000000001, 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98059 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441760+0000", 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001082185 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441780+0000", 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0463e-05 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441829+0000", 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8763000000000001e-05 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441849+0000", 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9423e-05 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574173+0000", 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13232460400000001 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574367+0000", 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019403000000000001 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574593+0000", 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000225372 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.114 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98060 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3833856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441880+0000", 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.634863777000007, 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13320725899999999, 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98060 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.115 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441881+0000", 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6160000000000001e-06 2024-09-18T01:00:04.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441889+0000", 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8439999999999998e-06 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441909+0000", 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9856000000000001e-05 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441925+0000", 2024-09-18T01:00:04.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6039999999999999e-05 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574669+0000", 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13274412799999999 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574860+0000", 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019038599999999999 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575087+0000", 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022738899999999999 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98061 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4120576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441947+0000", 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.634797020999997, 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13362664099999999, 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.119 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98061 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441948+0000", 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3200000000000001e-06 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441954+0000", 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3099999999999997e-06 2024-09-18T01:00:04.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441981+0000", 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7095999999999999e-05 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441998+0000", 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6237000000000001e-05 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575165+0000", 2024-09-18T01:00:04.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13316721200000001 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575355+0000", 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000190218 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575573+0000", 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021824799999999999 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98062 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4407296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.442019+0000", 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.634725051999993, 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13404179099999999, 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98062 2024-09-18T01:00:04.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445789+0000", 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037705569999999999 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445805+0000", 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.558e-05 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445846+0000", 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1066000000000003e-05 2024-09-18T01:00:04.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445871+0000", 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5072999999999999e-05 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575651+0000", 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12977999600000001 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575835+0000", 2024-09-18T01:00:04.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000183931 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576060+0000", 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022558800000000001 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98064 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2115584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.445983+0000", 2024-09-18T01:00:04.128 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.630760613000007, 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13103894799999999, 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98064 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445985+0000", 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445991+0000", 2024-09-18T01:00:04.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446014+0000", 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3056000000000001e-05 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446030+0000", 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5662999999999999e-05 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576622+0000", 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13059273900000001 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576805+0000", 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018304899999999999 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577022+0000", 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216481 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98065 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2402304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446050+0000", 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.630693496999996, 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131457463, 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98065 2024-09-18T01:00:04.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446051+0000", 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1260000000000001e-06 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446057+0000", 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2040000000000002e-06 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446077+0000", 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446092+0000", 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5124e-05 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577098+0000", 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13100595100000001 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577284+0000", 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000186152 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577508+0000", 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022314000000000001 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98066 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2689024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446112+0000", 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.630631291, 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13189522100000001, 2024-09-18T01:00:04.137 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98066 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446113+0000", 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0240000000000001e-06 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446120+0000", 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-18T01:00:04.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446137+0000", 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7479e-05 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446152+0000", 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4633e-05 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577587+0000", 2024-09-18T01:00:04.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13143572100000001 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577786+0000", 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019872999999999999 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578008+0000", 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022141399999999999 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98067 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2975744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446170+0000", 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.630573627999993, 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132319567, 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98067 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.142 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446171+0000", 2024-09-18T01:00:04.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0860000000000001e-06 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446178+0000", 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699999999999997e-06 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446196+0000", 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7989999999999999e-05 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446213+0000", 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7493999999999999e-05 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578083+0000", 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13186995000000001 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578266+0000", 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018321099999999999 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578490+0000", 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022316600000000001 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98068 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3262464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446231+0000", 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.630512722000006, 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273661, 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98068 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446232+0000", 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.187e-06 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446238+0000", 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446257+0000", 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8732999999999998e-05 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446272+0000", 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5532999999999998e-05 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578569+0000", 2024-09-18T01:00:04.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132296631 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578749+0000", 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017978300000000001 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578967+0000", 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218736 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98069 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3549184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446291+0000", 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.630453048999996, 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13311771, 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98069 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446292+0000", 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446298+0000", 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4470000000000003e-06 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446340+0000", 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1705999999999997e-05 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446358+0000", 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7873999999999999e-05 2024-09-18T01:00:04.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579072+0000", 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13271396399999999 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579213+0000", 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014133900000000001 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579408+0000", 2024-09-18T01:00:04.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019492700000000001 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98070 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3835904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446384+0000", 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.630359605999999, 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133517145, 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.155 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98070 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446385+0000", 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446393+0000", 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2629999999999997e-06 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446413+0000", 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9930000000000001e-05 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446428+0000", 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5336e-05 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579494+0000", 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13306637399999999 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579682+0000", 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018740900000000001 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579901+0000", 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021946300000000001 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98071 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446447+0000", 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.630296439999995, 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133931615, 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98071 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449904+0000", 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003456822 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449945+0000", 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0896000000000003e-05 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449971+0000", 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5996999999999999e-05 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449986+0000", 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.49e-05 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579978+0000", 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.129992723 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580157+0000", 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179006 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580379+0000", 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022127099999999999 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.163 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98073 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4696064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.450056+0000", 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.626687313000005, 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13113503100000001, 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98073 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450057+0000", 2024-09-18T01:00:04.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450060+0000", 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5329999999999999e-06 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450072+0000", 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1919999999999999e-05 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450080+0000", 2024-09-18T01:00:04.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9500000000000001e-06 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580951+0000", 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.130871128 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581057+0000", 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010501399999999999 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:04.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:04.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581191+0000", 2024-09-18T01:00:04.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013479900000000001 2024-09-18T01:00:04.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.168 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:04.168 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:04.168 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:04.170 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:00:04.170 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:04.169+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:04.270 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:04.269+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:04.303 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:04.303 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:04.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.303 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160910 3.1b 3.104fa77b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-18T01:00:04.303 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:03.953187+0000", 2024-09-18T01:00:04.303 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.35092468199999999, 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.35096855599999999, 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "waiting for peered", 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160910 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.953187+0000", 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.953187+0000", 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.953191+0000", 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.287e-06 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.953203+0000", 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2802999999999999e-05 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.953240+0000", 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6977000000000002e-05 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.953265+0000", 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5109e-05 2024-09-18T01:00:04.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.953484+0000", 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021837899999999999 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.953507+0000", 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3553e-05 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.969465+0000", 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015958014 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.969495+0000", 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9187e-05 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160911 3.1b 3.104fa77b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-18T01:00:04.308 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:03.956730+0000", 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.34738170800000001, 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.34756701099999998, 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "waiting for peered", 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160911 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.956730+0000", 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.956730+0000", 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.956734+0000", 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.264e-06 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.956746+0000", 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.273e-05 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.956786+0000", 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9252999999999998e-05 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.956809+0000", 2024-09-18T01:00:04.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3142999999999998e-05 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.956938+0000", 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012911899999999999 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.956948+0000", 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0226e-05 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.969595+0000", 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012647049 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:03.969619+0000", 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4277000000000002e-05 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: ], 2024-09-18T01:00:04.313 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 2 2024-09-18T01:00:04.314 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:04.318 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:00:04.371 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:04.369+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:04.441 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:04.441 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:04.442 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:04.442 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:04.455 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:00:04.500 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:04.501+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:04.573 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:04.573+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:04.579 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:04.579 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:04.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:04.579 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:04.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.579 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:04.579 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 135.80386951700001, 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T01:00:04.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T01:00:04.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T01:00:04.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87406 3.19 3:9d734bda:::rbd_data.117c8934add3.000000000000007c:head [stat,delete] snapc 1f=[1f] RETRY=8 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 76.155946395000001, 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097024104999999999, 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87406 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425532+0000", 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027458309999999998 2024-09-18T01:00:04.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425537+0000", 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2839999999999997e-06 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425551+0000", 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3682999999999999e-05 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425559+0000", 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5329999999999997e-06 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515035+0000", 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.089476411000000006 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515168+0000", 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013309699999999999 2024-09-18T01:00:04.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515475+0000", 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030662199999999999 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519553+0000", 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040781259999999996 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519665+0000", 2024-09-18T01:00:04.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112669 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519748+0000", 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2270999999999999e-05 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519810+0000", 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2577999999999996e-05 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.591 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87414 3.19 3:9c145b25:::rbd_data.117c8934add3.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 76.153122855999996, 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095886693999999995, 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.592 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87414 2024-09-18T01:00:04.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.593 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:04.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:04.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:04.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2024-09-18T01:00:04.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425618+0000", 2024-09-18T01:00:04.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:00:04.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425625+0000", 2024-09-18T01:00:04.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0870000000000001e-06 2024-09-18T01:00:04.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425630+0000", 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.262e-06 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518417+0000", 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092787816999999995 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518593+0000", 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017569100000000001 2024-09-18T01:00:04.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518939+0000", 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034572299999999998 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521320+0000", 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023809619999999999 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:04.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521343+0000", 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3317000000000001e-05 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521422+0000", 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480999999999996e-05 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521496+0000", 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4909999999999999e-05 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.600 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99159 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 393216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.209466505999998, 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096008205999999999, 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.601 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99159 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369409+0000", 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014342599999999999 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369415+0000", 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.31e-06 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369426+0000", 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1677e-05 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369435+0000", 2024-09-18T01:00:04.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.464949+0000", 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095513718999999997 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465099+0000", 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014984300000000001 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465274+0000", 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017554499999999999 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99160 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2336768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.209281937, 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096064584999999994, 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.605 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99160 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369564+0000", 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113309 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369569+0000", 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5269999999999998e-06 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369582+0000", 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2952999999999999e-05 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369592+0000", 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0939999999999997e-06 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465328+0000", 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095736285000000004 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465403+0000", 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5498000000000001e-05 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465515+0000", 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111919 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99161 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 413696~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:04.609 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.209124308, 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096302270999999995, 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99161 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:04.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373792+0000", 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004184063 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373801+0000", 2024-09-18T01:00:04.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3729999999999993e-06 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373818+0000", 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7280000000000001e-05 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373829+0000", 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0540000000000001e-05 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465596+0000", 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.091767910999999994 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465740+0000", 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143683 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465911+0000", 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000170421 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.613 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99172 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2459648~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.204528964999994, 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095842035000000006, 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99172 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.614 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9299999999999997e-07 2024-09-18T01:00:04.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374207+0000", 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8899999999999999e-06 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374216+0000", 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3669999999999997e-06 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374223+0000", 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9759999999999998e-06 2024-09-18T01:00:04.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469763+0000", 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095540307000000005 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469894+0000", 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013135600000000001 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470046+0000", 2024-09-18T01:00:04.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015144600000000001 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99173 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2480128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.204501405000002, 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096195265000000002, 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.618 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99173 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374232+0000", 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4600000000000004e-07 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374235+0000", 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8569999999999999e-06 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374243+0000", 2024-09-18T01:00:04.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5930000000000006e-06 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374250+0000", 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9330000000000004e-06 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470113+0000", 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095862952000000001 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470259+0000", 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014534300000000001 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470426+0000", 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016784100000000001 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99174 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2500608~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.204473098999998, 2024-09-18T01:00:04.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096540567999999993, 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99174 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374260+0000", 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2600000000000002e-07 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374263+0000", 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8770000000000002e-06 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374272+0000", 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6070000000000008e-06 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374279+0000", 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8430000000000004e-06 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470492+0000", 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096213513000000001 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470632+0000", 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013954899999999999 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470800+0000", 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016845299999999999 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.626 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99175 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 536576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.204445203000006, 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096867154999999996, 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.627 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99175 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374288+0000", 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9999999999999997e-07 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374291+0000", 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.92e-06 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374300+0000", 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5939999999999994e-06 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374307+0000", 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470858+0000", 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096551085999999994 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470990+0000", 2024-09-18T01:00:04.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013223900000000001 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471155+0000", 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016463299999999999 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99176 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2521088~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.204417253000003, 2024-09-18T01:00:04.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097204272999999994, 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99176 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:04.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374316+0000", 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1999999999999999e-07 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374336+0000", 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9879999999999999e-05 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374346+0000", 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0353999999999999e-05 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374354+0000", 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7130000000000005e-06 2024-09-18T01:00:04.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471210+0000", 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096856271999999993 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471350+0000", 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013940900000000001 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471520+0000", 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000169925 2024-09-18T01:00:04.635 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99177 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 557056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.204368389999999, 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097507736999999997, 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.636 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99177 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377919+0000", 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035553070000000002 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377927+0000", 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4569999999999999e-06 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377943+0000", 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5563e-05 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377953+0000", 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0413e-05 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471578+0000", 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093625036999999994 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471710+0000", 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013170900000000001 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471872+0000", 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162251 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99183 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 618496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.200580353999996, 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095979540000000002, 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99183 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:04.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378153+0000", 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.23e-07 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378156+0000", 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9340000000000002e-06 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378164+0000", 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4630000000000001e-06 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378173+0000", 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8130000000000002e-06 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473851+0000", 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678257000000003 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473982+0000", 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000130875 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474132+0000", 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014947499999999999 2024-09-18T01:00:04.644 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99184 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2603008~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.200523812, 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096289871999999999, 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.645 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99184 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378210+0000", 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2999999999999999e-07 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378213+0000", 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378223+0000", 2024-09-18T01:00:04.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378230+0000", 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5830000000000001e-06 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474192+0000", 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961742000000003 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474324+0000", 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000132099 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474499+0000", 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174551 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99185 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 638976~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:04.649 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.200489384999997, 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096618187999999994, 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99185 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378244+0000", 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378248+0000", 2024-09-18T01:00:04.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5130000000000001e-06 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378256+0000", 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8429999999999999e-06 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378263+0000", 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9870000000000002e-06 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474554+0000", 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096290693999999996 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474692+0000", 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013761900000000001 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474861+0000", 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016967499999999999 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.653 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99186 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 659456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.200459835000004, 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096958847000000001, 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.654 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99186 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1699999999999997e-07 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378277+0000", 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0800000000000002e-06 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378285+0000", 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5699999999999993e-06 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378292+0000", 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9600000000000003e-06 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474924+0000", 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096632207999999997 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475067+0000", 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143175 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475232+0000", 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016413699999999999 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99187 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 679936~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.200431562000006, 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097293710000000005, 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.658 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99187 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378302+0000", 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378305+0000", 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.066e-06 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378315+0000", 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0443e-05 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378323+0000", 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3239999999999997e-06 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475291+0000", 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096968726000000005 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475430+0000", 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013847499999999999 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475595+0000", 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016497899999999999 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99188 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 700416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.662 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.200400725999998, 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097626477000000003, 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99188 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381945+0000", 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036127569999999999 2024-09-18T01:00:04.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381960+0000", 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.554e-05 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381977+0000", 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6733000000000001e-05 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381987+0000", 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.031e-05 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475651+0000", 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093664175000000002 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475789+0000", 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000137181 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475958+0000", 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016978100000000001 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99195 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 761856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.196527920999998, 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961833999999996, 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.667 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99195 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382208+0000", 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0469999999999998e-06 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382217+0000", 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5159999999999994e-06 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382224+0000", 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.64e-06 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.477889+0000", 2024-09-18T01:00:04.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095664955999999995 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478012+0000", 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012299899999999999 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478166+0000", 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015497899999999999 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99196 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 88064~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.196499990999996, 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096312801000000003, 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99196 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.672 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385636+0000", 2024-09-18T01:00:04.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034037450000000001 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385642+0000", 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9900000000000002e-06 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385659+0000", 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6753e-05 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:04.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385669+0000", 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9170000000000001e-06 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478225+0000", 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092555514000000005 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478366+0000", 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014198699999999999 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:04.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478545+0000", 2024-09-18T01:00:04.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017889499999999999 2024-09-18T01:00:04.676 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.676 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.676 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.676 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:04.676 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:04.676 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:04.678 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:00:04.678 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:04.677+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:04.778 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:04.777+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:04.849 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:04.849 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:04.849 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:04.850 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:04.863 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:00:04.879 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:04.877+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:04.963 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:00:04.989 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:04.989 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:04.989 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:04.989 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:04.990 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:05.004 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:00:05.010 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:05.009+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:05.080 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:05.081+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:05.126 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:05.126 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:05.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:05.126 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:05.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.126 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 135.714998702, 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T01:00:05.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T01:00:05.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.132 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125622 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 344064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.450863535000003, 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096365904000000002, 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125622 2024-09-18T01:00:05.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673347+0000", 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7599999999999997e-07 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673354+0000", 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.486e-06 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673361+0000", 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5520000000000001e-06 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673365+0000", 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7129999999999996e-06 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768923+0000", 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095557673999999995 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768994+0000", 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0847999999999997e-05 2024-09-18T01:00:05.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769712+0000", 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000718055 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125623 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.450839121000001, 2024-09-18T01:00:05.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096098244999999999, 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125623 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8200000000000001e-07 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673372+0000", 2024-09-18T01:00:05.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5519999999999999e-06 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673376+0000", 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.512e-06 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673379+0000", 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1580000000000001e-06 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769200+0000", 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095820946000000004 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769317+0000", 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117003 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769469+0000", 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015179200000000001 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125624 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.450827257, 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101719451, 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125624 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673383+0000", 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.23e-07 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673384+0000", 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3060000000000001e-06 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673387+0000", 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:00:05.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673390+0000", 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769495+0000", 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096104599999999998 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769571+0000", 2024-09-18T01:00:05.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6089999999999998e-05 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775102+0000", 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055311859999999996 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125625 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.450816213000003, 2024-09-18T01:00:05.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102242338, 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125625 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673394+0000", 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673395+0000", 2024-09-18T01:00:05.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2839999999999999e-06 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673398+0000", 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1379999999999999e-06 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673401+0000", 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7149999999999998e-06 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775140+0000", 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101739127 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775212+0000", 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1984999999999996e-05 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775636+0000", 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000423752 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.150 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125627 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.450805295999999, 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10248196399999999, 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125627 2024-09-18T01:00:05.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673405+0000", 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0699999999999998e-07 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673406+0000", 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2130000000000001e-06 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673409+0000", 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3400000000000002e-06 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673412+0000", 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7360000000000001e-06 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775665+0000", 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102253552 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775734+0000", 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8019e-05 2024-09-18T01:00:05.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775886+0000", 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015279700000000001 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125628 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.450794197999997, 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.1031562, 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.155 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125628 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673416+0000", 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.91e-07 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673418+0000", 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:00:05.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673423+0000", 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6149999999999996e-06 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673428+0000", 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1460000000000004e-06 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776149+0000", 2024-09-18T01:00:05.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10272025899999999 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776218+0000", 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9207000000000003e-05 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776572+0000", 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035375599999999999 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.159 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125629 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.450774647000003, 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103390994, 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.160 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125629 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673438+0000", 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4310000000000001e-06 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673446+0000", 2024-09-18T01:00:05.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0290000000000005e-06 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673451+0000", 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3630000000000002e-06 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776600+0000", 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10314894500000001 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776668+0000", 2024-09-18T01:00:05.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7963e-05 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776826+0000", 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015784499999999999 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125631 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:05.164 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.450751396999998, 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.104002186, 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125631 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673459+0000", 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673461+0000", 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.351e-06 2024-09-18T01:00:05.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673467+0000", 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3389999999999998e-06 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673472+0000", 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777062+0000", 2024-09-18T01:00:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103589771 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777130+0000", 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8065999999999995e-05 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777460+0000", 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000330265 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125632 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.450730608999997, 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10711156300000001, 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125632 2024-09-18T01:00:05.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:05.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6100000000000002e-07 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673482+0000", 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1210000000000001e-06 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673488+0000", 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1659999999999999e-06 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673493+0000", 2024-09-18T01:00:05.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.989e-06 2024-09-18T01:00:05.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777489+0000", 2024-09-18T01:00:05.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10399652600000001 2024-09-18T01:00:05.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777565+0000", 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6219999999999999e-05 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780591+0000", 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030251800000000001 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125633 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.173 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.450709846999999, 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10735947699999999, 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125633 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673503+0000", 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673509+0000", 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3969999999999999e-06 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673515+0000", 2024-09-18T01:00:05.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7640000000000002e-06 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780622+0000", 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.107107533 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780692+0000", 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9735999999999998e-05 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780859+0000", 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016721999999999999 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126363 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.412855706000002, 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10567230599999999, 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.178 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126363 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711355+0000", 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9999999999999995e-07 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711360+0000", 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:00:05.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711380+0000", 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9769e-05 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711396+0000", 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5947000000000002e-05 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817011+0000", 2024-09-18T01:00:05.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.105615185 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817026+0000", 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5430999999999998e-05 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126376 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.402279421000003, 2024-09-18T01:00:05.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097651948000000002, 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126376 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721931+0000", 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721933+0000", 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721939+0000", 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721944+0000", 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0089999999999998e-06 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817590+0000", 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095646089000000004 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.819582+0000", 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019924669999999999 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126427 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.186 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.402258887999999, 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095671891999999994, 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126427 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721954+0000", 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4700000000000001e-06 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721960+0000", 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4629999999999998e-06 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721965+0000", 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:00:05.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817614+0000", 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095649026999999998 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817623+0000", 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4780000000000008e-06 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126428 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.402236701, 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095683878999999999, 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126428 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.191 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721976+0000", 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4279999999999999e-06 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721983+0000", 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7499999999999997e-06 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721988+0000", 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4349999999999996e-06 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817644+0000", 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095656525000000006 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817657+0000", 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2312e-05 2024-09-18T01:00:05.194 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126429 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.402213660999998, 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095687929000000005, 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.195 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126429 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721999+0000", 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.649e-06 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722006+0000", 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8299999999999998e-06 2024-09-18T01:00:05.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722011+0000", 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6970000000000001e-06 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817678+0000", 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095667143999999996 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817684+0000", 2024-09-18T01:00:05.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2149999999999998e-06 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126430 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.402191537999997, 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095682481999999999, 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.199 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126430 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722019+0000", 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722021+0000", 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.419e-06 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722027+0000", 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0750000000000003e-06 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722033+0000", 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817695+0000", 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095662607999999996 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817701+0000", 2024-09-18T01:00:05.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1359999999999996e-06 2024-09-18T01:00:05.203 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.203 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.204 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.204 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126431 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:05.204 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:05.204 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.402169635, 2024-09-18T01:00:05.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678416000000002, 2024-09-18T01:00:05.204 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.204 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126431 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722043+0000", 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:00:05.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722049+0000", 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3219999999999998e-06 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722054+0000", 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817715+0000", 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095660429000000005 2024-09-18T01:00:05.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817718+0000", 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160401 3.0 3:0686cf5d:::rbd_data.117cf66ad0cc.0000000000000001:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.0530072050000001, 2024-09-18T01:00:05.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.89130826799999996, 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160401 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:05.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071217+0000", 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4924e-05 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071229+0000", 2024-09-18T01:00:05.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.166e-05 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071272+0000", 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2506000000000003e-05 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071294+0000", 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2789000000000002e-05 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071664+0000", 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036966100000000002 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071694+0000", 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0207000000000001e-05 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.955396+0000", 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88370225899999999 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.955864+0000", 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000467997 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.956147+0000", 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028220399999999998 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.960924+0000", 2024-09-18T01:00:05.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047771760000000002 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962456+0000", 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001532158 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962485+0000", 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8758000000000001e-05 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962511+0000", 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5969000000000002e-05 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160404 3.0 3:048cdd79:::rbd_data.117cf66ad0cc.0000000000000004:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.0528301520000003, 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.89253343799999996, 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:05.215 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160404 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071381+0000", 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7829999999999999e-06 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071390+0000", 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.02e-06 2024-09-18T01:00:05.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071417+0000", 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6798999999999999e-05 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071434+0000", 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6983e-05 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071769+0000", 2024-09-18T01:00:05.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033536500000000001 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071788+0000", 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961054+0000", 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88926583000000003 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:05.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961171+0000", 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116468 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961288+0000", 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117759 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962832+0000", 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001543677 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:05.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963849+0000", 2024-09-18T01:00:05.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010164659999999999 2024-09-18T01:00:05.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:05.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963878+0000", 2024-09-18T01:00:05.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9697e-05 2024-09-18T01:00:05.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:05.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:05.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:05.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963913+0000", 2024-09-18T01:00:05.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4628000000000001e-05 2024-09-18T01:00:05.222 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.222 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.222 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.222 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:05.222 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:05.222 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:05.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:00:05.224 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:05.221+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:05.324 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:05.321+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:05.399 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:05.399 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:05.399 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:05.399 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:05.413 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:00:05.421 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:00:05.466 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:05.465+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:05.526 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:05.525+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:05.531 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:05.531 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:05.531 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:05.531 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:05.531 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:05.547 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:00:05.575 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:05.575 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:05.592 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:05.592 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:05.611 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:05.611 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:05.630 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:05.630 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:05.632 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:05.633+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:05.648 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:05.669 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:05.669 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:05.689 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:05.689 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:05.710 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:05.710 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:05.710 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:05.724 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:00:05.728 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:05.725+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:05.829 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:05.825+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:05.891 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:05.891 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:05.905 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:00:05.952 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:05.953+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T01:00:06.031 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:06.029+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:06.048 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:06.049 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:06.078 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:00:06.131 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:06.129+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:06.232 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:06.229+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:06.273 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:06.273 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:06.289 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:00:06.338 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:06.337+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:06.428 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:06.429 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:06.433 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:06.429+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:06.440 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:00:06.534 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:06.533+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:06.607 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:06.607 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:06.621 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:00:06.635 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:06.633+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:06.736 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:06.733+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:06.785 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:06.785 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:06.799 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:00:06.842 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:06.841+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:06.938 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:06.937+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:06.980 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:06.980 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:06.997 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:00:07.042 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:07.041+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:07.140 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:07.137+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:07.180 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:07.180 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:07.195 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:00:07.241 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:07.241+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:07.343 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:07.341+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:07.359 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:07.359 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:07.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:00:07.444 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:07.441+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:07.501 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:07.501 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:07.516 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:00:07.544 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:07.545+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:07.645 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:07.645+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:07.685 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:07.685 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:07.700 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:00:07.730 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:00:07.746 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:07.745+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:07.829 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:07.829 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:07.844 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:00:07.846 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:07.845+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:07.890 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:07.908 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:07.908 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:07.926 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:07.927 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:07.945 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:07.946 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:07.946 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:07.945+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:07.966 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:07.966 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:07.968 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:07.968 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:07.968 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:07.968 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:07.985 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:00:07.988 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:07.988 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:07.996 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:00:08.011 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:08.045 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:08.045 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:08.047 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:08.045+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:08.106 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:08.106 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:08.106 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:08.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:00:08.148 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:08.149+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:08.242 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:08.242 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:08.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:08.242 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:08.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.242 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98052 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4179968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.242 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440036+0000", 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.798172855000004, 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131360583, 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98052 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440037+0000", 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:00:08.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440040+0000", 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7159999999999999e-06 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440048+0000", 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7169999999999997e-06 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440056+0000", 2024-09-18T01:00:08.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1860000000000003e-06 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.570989+0000", 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13093384999999999 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571175+0000", 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018504799999999999 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571397+0000", 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022245899999999999 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98053 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4466688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440064+0000", 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.798144985999997, 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13182195299999999, 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.247 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98053 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440065+0000", 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440068+0000", 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6699999999999998e-06 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440076+0000", 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7069999999999998e-06 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440083+0000", 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9929999999999998e-06 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571475+0000", 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13139158100000001 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571657+0000", 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018247099999999999 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571886+0000", 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022892400000000001 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.251 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98054 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2113536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.252 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440092+0000", 2024-09-18T01:00:08.252 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.798117125999994, 2024-09-18T01:00:08.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13228911600000001, 2024-09-18T01:00:08.252 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.252 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.252 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98054 2024-09-18T01:00:08.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.253 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T01:00:08.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440093+0000", 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440096+0000", 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440105+0000", 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7739999999999993e-06 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440112+0000", 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2130000000000002e-06 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571970+0000", 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13185812199999999 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572159+0000", 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000188665 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572381+0000", 2024-09-18T01:00:08.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022273200000000001 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98055 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2400256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440122+0000", 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.798086945999998, 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273966600000001, 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.257 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98055 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T01:00:08.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440123+0000", 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440126+0000", 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440137+0000", 2024-09-18T01:00:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0543000000000001e-05 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440144+0000", 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572457+0000", 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132312807 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572636+0000", 2024-09-18T01:00:08.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882299999999999 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572862+0000", 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022645200000000001 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98056 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2686976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440153+0000", 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.798056368999994, 2024-09-18T01:00:08.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13308801200000001, 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98056 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440154+0000", 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5300000000000004e-07 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440156+0000", 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.903e-06 2024-09-18T01:00:08.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440165+0000", 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7600000000000008e-06 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440174+0000", 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6969999999999999e-06 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572960+0000", 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132785984 2024-09-18T01:00:08.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573087+0000", 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012728699999999999 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573241+0000", 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153728 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98057 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2973696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440182+0000", 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.798026776, 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13357376900000001, 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98057 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440183+0000", 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-18T01:00:08.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440186+0000", 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8700000000000001e-06 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440195+0000", 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7099999999999996e-06 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440202+0000", 2024-09-18T01:00:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2429999999999999e-06 2024-09-18T01:00:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573338+0000", 2024-09-18T01:00:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13313578400000001 2024-09-18T01:00:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573526+0000", 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018780399999999999 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573756+0000", 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230755 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98058 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440210+0000", 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.797998892999999, 2024-09-18T01:00:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13387749400000001, 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98058 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T01:00:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440627+0000", 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041660699999999999 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440634+0000", 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.596e-06 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440650+0000", 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6164e-05 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440660+0000", 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.023e-05 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573833+0000", 2024-09-18T01:00:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13317317000000001 2024-09-18T01:00:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573945+0000", 2024-09-18T01:00:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111731 2024-09-18T01:00:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574088+0000", 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000142996 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98059 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3547136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440678+0000", 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.797531546000002, 2024-09-18T01:00:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13391484000000001, 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98059 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441760+0000", 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001082185 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441780+0000", 2024-09-18T01:00:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0463e-05 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441829+0000", 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8763000000000001e-05 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441849+0000", 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9423e-05 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574173+0000", 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13232460400000001 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574367+0000", 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019403000000000001 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574593+0000", 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000225372 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98060 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3833856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441880+0000", 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.796329464999999, 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13320725899999999, 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98060 2024-09-18T01:00:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441881+0000", 2024-09-18T01:00:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6160000000000001e-06 2024-09-18T01:00:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441889+0000", 2024-09-18T01:00:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8439999999999998e-06 2024-09-18T01:00:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441909+0000", 2024-09-18T01:00:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9856000000000001e-05 2024-09-18T01:00:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441925+0000", 2024-09-18T01:00:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6039999999999999e-05 2024-09-18T01:00:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574669+0000", 2024-09-18T01:00:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13274412799999999 2024-09-18T01:00:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574860+0000", 2024-09-18T01:00:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019038599999999999 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575087+0000", 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022738899999999999 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98061 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4120576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441947+0000", 2024-09-18T01:00:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.796262709000004, 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13362664099999999, 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98061 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441948+0000", 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3200000000000001e-06 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441954+0000", 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3099999999999997e-06 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441981+0000", 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7095999999999999e-05 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441998+0000", 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6237000000000001e-05 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575165+0000", 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13316721200000001 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575355+0000", 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000190218 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575573+0000", 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021824799999999999 2024-09-18T01:00:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98062 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4407296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.442019+0000", 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.79619074, 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13404179099999999, 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98062 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445789+0000", 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037705569999999999 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445805+0000", 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.558e-05 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445846+0000", 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1066000000000003e-05 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445871+0000", 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5072999999999999e-05 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575651+0000", 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12977999600000001 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575835+0000", 2024-09-18T01:00:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000183931 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576060+0000", 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022558800000000001 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98064 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2115584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.445983+0000", 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.792226300999999, 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13103894799999999, 2024-09-18T01:00:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98064 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445985+0000", 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445991+0000", 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T01:00:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446014+0000", 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3056000000000001e-05 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446030+0000", 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5662999999999999e-05 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576622+0000", 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13059273900000001 2024-09-18T01:00:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576805+0000", 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018304899999999999 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577022+0000", 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216481 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98065 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2402304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446050+0000", 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.792159185000003, 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131457463, 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98065 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446051+0000", 2024-09-18T01:00:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1260000000000001e-06 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446057+0000", 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2040000000000002e-06 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446077+0000", 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446092+0000", 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5124e-05 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577098+0000", 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13100595100000001 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577284+0000", 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000186152 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577508+0000", 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022314000000000001 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98066 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2689024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446112+0000", 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.792096978999993, 2024-09-18T01:00:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13189522100000001, 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98066 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446113+0000", 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0240000000000001e-06 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446120+0000", 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446137+0000", 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7479e-05 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446152+0000", 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4633e-05 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577587+0000", 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13143572100000001 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577786+0000", 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019872999999999999 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578008+0000", 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022141399999999999 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98067 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2975744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446170+0000", 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.792039316, 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132319567, 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98067 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446171+0000", 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0860000000000001e-06 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446178+0000", 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699999999999997e-06 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446196+0000", 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7989999999999999e-05 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446213+0000", 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7493999999999999e-05 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578083+0000", 2024-09-18T01:00:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13186995000000001 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578266+0000", 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018321099999999999 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578490+0000", 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022316600000000001 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98068 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3262464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446231+0000", 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.791978409999999, 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273661, 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98068 2024-09-18T01:00:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446232+0000", 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.187e-06 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446238+0000", 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446257+0000", 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8732999999999998e-05 2024-09-18T01:00:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446272+0000", 2024-09-18T01:00:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5532999999999998e-05 2024-09-18T01:00:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578569+0000", 2024-09-18T01:00:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132296631 2024-09-18T01:00:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578749+0000", 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017978300000000001 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578967+0000", 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218736 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98069 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3549184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446291+0000", 2024-09-18T01:00:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.791918737000003, 2024-09-18T01:00:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13311771, 2024-09-18T01:00:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98069 2024-09-18T01:00:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T01:00:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T01:00:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446292+0000", 2024-09-18T01:00:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:00:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446298+0000", 2024-09-18T01:00:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4470000000000003e-06 2024-09-18T01:00:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446340+0000", 2024-09-18T01:00:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1705999999999997e-05 2024-09-18T01:00:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446358+0000", 2024-09-18T01:00:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7873999999999999e-05 2024-09-18T01:00:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579072+0000", 2024-09-18T01:00:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13271396399999999 2024-09-18T01:00:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579213+0000", 2024-09-18T01:00:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014133900000000001 2024-09-18T01:00:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579408+0000", 2024-09-18T01:00:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019492700000000001 2024-09-18T01:00:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98070 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3835904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446384+0000", 2024-09-18T01:00:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.791825294000006, 2024-09-18T01:00:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133517145, 2024-09-18T01:00:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98070 2024-09-18T01:00:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T01:00:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T01:00:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446385+0000", 2024-09-18T01:00:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-09-18T01:00:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446393+0000", 2024-09-18T01:00:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2629999999999997e-06 2024-09-18T01:00:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446413+0000", 2024-09-18T01:00:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9930000000000001e-05 2024-09-18T01:00:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446428+0000", 2024-09-18T01:00:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5336e-05 2024-09-18T01:00:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579494+0000", 2024-09-18T01:00:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13306637399999999 2024-09-18T01:00:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579682+0000", 2024-09-18T01:00:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018740900000000001 2024-09-18T01:00:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579901+0000", 2024-09-18T01:00:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021946300000000001 2024-09-18T01:00:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98071 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446447+0000", 2024-09-18T01:00:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.791762128000002, 2024-09-18T01:00:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133931615, 2024-09-18T01:00:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98071 2024-09-18T01:00:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449904+0000", 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003456822 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449945+0000", 2024-09-18T01:00:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0896000000000003e-05 2024-09-18T01:00:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449971+0000", 2024-09-18T01:00:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5996999999999999e-05 2024-09-18T01:00:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449986+0000", 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.49e-05 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579978+0000", 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.129992723 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580157+0000", 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179006 2024-09-18T01:00:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580379+0000", 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022127099999999999 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98073 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4696064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.450056+0000", 2024-09-18T01:00:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.788153000999998, 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13113503100000001, 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98073 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T01:00:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T01:00:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450057+0000", 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450060+0000", 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5329999999999999e-06 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450072+0000", 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1919999999999999e-05 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450080+0000", 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9500000000000001e-06 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580951+0000", 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.130871128 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581057+0000", 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010501399999999999 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581191+0000", 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013479900000000001 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:08.348 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:08.350 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:00:08.350 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:08.349+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:08.451 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:08.449+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:08.501 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:08.501 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:08.501 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:08.501 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:08.516 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:00:08.560 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:08.561+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:08.639 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:08.640 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:08.640 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:08.640 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:08.640 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:08.653 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:00:08.653 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:08.653+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:08.753 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:08.753+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:08.824 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:08.824 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:08.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:08.824 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:08.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.824 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:08.824 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:08.824 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 140.04804717499999, 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T01:00:08.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T01:00:08.830 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87406 3.19 3:9d734bda:::rbd_data.117c8934add3.000000000000007c:head [stat,delete] snapc 1f=[1f] RETRY=8 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.400124052999999, 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097024104999999999, 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.831 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87406 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425532+0000", 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027458309999999998 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425537+0000", 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2839999999999997e-06 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425551+0000", 2024-09-18T01:00:08.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3682999999999999e-05 2024-09-18T01:00:08.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425559+0000", 2024-09-18T01:00:08.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5329999999999997e-06 2024-09-18T01:00:08.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515035+0000", 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.089476411000000006 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515168+0000", 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013309699999999999 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515475+0000", 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030662199999999999 2024-09-18T01:00:08.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519553+0000", 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040781259999999996 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519665+0000", 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112669 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:08.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519748+0000", 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2270999999999999e-05 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519810+0000", 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2577999999999996e-05 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87414 3.19 3:9c145b25:::rbd_data.117c8934add3.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T01:00:08.837 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.397300513999994, 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095886693999999995, 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87414 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2024-09-18T01:00:08.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425618+0000", 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425625+0000", 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0870000000000001e-06 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425630+0000", 2024-09-18T01:00:08.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.262e-06 2024-09-18T01:00:08.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518417+0000", 2024-09-18T01:00:08.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092787816999999995 2024-09-18T01:00:08.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518593+0000", 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017569100000000001 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518939+0000", 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034572299999999998 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521320+0000", 2024-09-18T01:00:08.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023809619999999999 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521343+0000", 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3317000000000001e-05 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521422+0000", 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480999999999996e-05 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521496+0000", 2024-09-18T01:00:08.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4909999999999999e-05 2024-09-18T01:00:08.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.844 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.844 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99159 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 393216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.844 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:08.844 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.453644163999996, 2024-09-18T01:00:08.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096008205999999999, 2024-09-18T01:00:08.844 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.844 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99159 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369409+0000", 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014342599999999999 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369415+0000", 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.31e-06 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369426+0000", 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1677e-05 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369435+0000", 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.464949+0000", 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095513718999999997 2024-09-18T01:00:08.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465099+0000", 2024-09-18T01:00:08.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014984300000000001 2024-09-18T01:00:08.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465274+0000", 2024-09-18T01:00:08.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017554499999999999 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99160 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2336768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.453459594999998, 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096064584999999994, 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.849 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99160 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:08.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369564+0000", 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113309 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369569+0000", 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5269999999999998e-06 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369582+0000", 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2952999999999999e-05 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369592+0000", 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0939999999999997e-06 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465328+0000", 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095736285000000004 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465403+0000", 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5498000000000001e-05 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465515+0000", 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111919 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99161 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 413696~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.453301965999998, 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096302270999999995, 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99161 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.854 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373792+0000", 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004184063 2024-09-18T01:00:08.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373801+0000", 2024-09-18T01:00:08.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3729999999999993e-06 2024-09-18T01:00:08.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373818+0000", 2024-09-18T01:00:08.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7280000000000001e-05 2024-09-18T01:00:08.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373829+0000", 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0540000000000001e-05 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465596+0000", 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.091767910999999994 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465740+0000", 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143683 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465911+0000", 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000170421 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99172 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2459648~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:08.858 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.448706623000007, 2024-09-18T01:00:08.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095842035000000006, 2024-09-18T01:00:08.859 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.859 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.859 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.859 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.859 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.860 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.860 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99172 2024-09-18T01:00:08.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.860 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9299999999999997e-07 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374207+0000", 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8899999999999999e-06 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374216+0000", 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3669999999999997e-06 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374223+0000", 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9759999999999998e-06 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469763+0000", 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095540307000000005 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469894+0000", 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013135600000000001 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470046+0000", 2024-09-18T01:00:08.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015144600000000001 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99173 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2480128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.448679063, 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096195265000000002, 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.864 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99173 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374232+0000", 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4600000000000004e-07 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374235+0000", 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8569999999999999e-06 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374243+0000", 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5930000000000006e-06 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374250+0000", 2024-09-18T01:00:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9330000000000004e-06 2024-09-18T01:00:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470113+0000", 2024-09-18T01:00:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095862952000000001 2024-09-18T01:00:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470259+0000", 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014534300000000001 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470426+0000", 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016784100000000001 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.868 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99174 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2500608~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.448650756999996, 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096540567999999993, 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99174 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374260+0000", 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2600000000000002e-07 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374263+0000", 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8770000000000002e-06 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374272+0000", 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6070000000000008e-06 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374279+0000", 2024-09-18T01:00:08.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8430000000000004e-06 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470492+0000", 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096213513000000001 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470632+0000", 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013954899999999999 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470800+0000", 2024-09-18T01:00:08.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016845299999999999 2024-09-18T01:00:08.873 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.873 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.873 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.873 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99175 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 536576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.873 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:08.873 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.448622861000004, 2024-09-18T01:00:08.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096867154999999996, 2024-09-18T01:00:08.874 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.874 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.874 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.874 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.874 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.874 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.874 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99175 2024-09-18T01:00:08.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.874 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374288+0000", 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9999999999999997e-07 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374291+0000", 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.92e-06 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374300+0000", 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5939999999999994e-06 2024-09-18T01:00:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374307+0000", 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470858+0000", 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096551085999999994 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470990+0000", 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013223900000000001 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471155+0000", 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016463299999999999 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99176 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2521088~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.448594911000001, 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097204272999999994, 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99176 2024-09-18T01:00:08.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374316+0000", 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1999999999999999e-07 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374336+0000", 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9879999999999999e-05 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374346+0000", 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0353999999999999e-05 2024-09-18T01:00:08.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374354+0000", 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7130000000000005e-06 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471210+0000", 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096856271999999993 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471350+0000", 2024-09-18T01:00:08.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013940900000000001 2024-09-18T01:00:08.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471520+0000", 2024-09-18T01:00:08.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000169925 2024-09-18T01:00:08.883 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.883 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.883 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.883 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99177 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 557056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.448546047999997, 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097507736999999997, 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99177 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377919+0000", 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035553070000000002 2024-09-18T01:00:08.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377927+0000", 2024-09-18T01:00:08.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4569999999999999e-06 2024-09-18T01:00:08.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377943+0000", 2024-09-18T01:00:08.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5563e-05 2024-09-18T01:00:08.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377953+0000", 2024-09-18T01:00:08.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0413e-05 2024-09-18T01:00:08.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471578+0000", 2024-09-18T01:00:08.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093625036999999994 2024-09-18T01:00:08.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471710+0000", 2024-09-18T01:00:08.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013170900000000001 2024-09-18T01:00:08.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471872+0000", 2024-09-18T01:00:08.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162251 2024-09-18T01:00:08.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.888 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99183 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 618496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.444758011999994, 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095979540000000002, 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.889 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99183 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378153+0000", 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.23e-07 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378156+0000", 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9340000000000002e-06 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378164+0000", 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4630000000000001e-06 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378173+0000", 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8130000000000002e-06 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473851+0000", 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678257000000003 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473982+0000", 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000130875 2024-09-18T01:00:08.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474132+0000", 2024-09-18T01:00:08.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014947499999999999 2024-09-18T01:00:08.893 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.893 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.893 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.893 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99184 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2603008~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.893 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.444701469999998, 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096289871999999999, 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99184 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:08.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378210+0000", 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2999999999999999e-07 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378213+0000", 2024-09-18T01:00:08.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378223+0000", 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378230+0000", 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5830000000000001e-06 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474192+0000", 2024-09-18T01:00:08.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961742000000003 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474324+0000", 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000132099 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474499+0000", 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174551 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.897 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99185 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 638976~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.444667042999995, 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096618187999999994, 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99185 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.898 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378244+0000", 2024-09-18T01:00:08.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378248+0000", 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5130000000000001e-06 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378256+0000", 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8429999999999999e-06 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378263+0000", 2024-09-18T01:00:08.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9870000000000002e-06 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474554+0000", 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096290693999999996 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474692+0000", 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013761900000000001 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474861+0000", 2024-09-18T01:00:08.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016967499999999999 2024-09-18T01:00:08.902 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.902 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.903 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99186 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 659456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.444637493000002, 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096958847000000001, 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.904 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99186 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1699999999999997e-07 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378277+0000", 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0800000000000002e-06 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378285+0000", 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5699999999999993e-06 2024-09-18T01:00:08.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378292+0000", 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9600000000000003e-06 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474924+0000", 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096632207999999997 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475067+0000", 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143175 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475232+0000", 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016413699999999999 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.908 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99187 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 679936~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.444609220000004, 2024-09-18T01:00:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097293710000000005, 2024-09-18T01:00:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.909 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99187 2024-09-18T01:00:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378302+0000", 2024-09-18T01:00:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:00:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378305+0000", 2024-09-18T01:00:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.066e-06 2024-09-18T01:00:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378315+0000", 2024-09-18T01:00:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0443e-05 2024-09-18T01:00:08.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378323+0000", 2024-09-18T01:00:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3239999999999997e-06 2024-09-18T01:00:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.914 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475291+0000", 2024-09-18T01:00:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096968726000000005 2024-09-18T01:00:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475430+0000", 2024-09-18T01:00:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013847499999999999 2024-09-18T01:00:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475595+0000", 2024-09-18T01:00:08.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016497899999999999 2024-09-18T01:00:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.917 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99188 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 700416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.918 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:08.918 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.444578383999996, 2024-09-18T01:00:08.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097626477000000003, 2024-09-18T01:00:08.918 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.918 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.918 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.919 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.919 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.919 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.919 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99188 2024-09-18T01:00:08.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.919 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:08.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:08.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381945+0000", 2024-09-18T01:00:08.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036127569999999999 2024-09-18T01:00:08.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381960+0000", 2024-09-18T01:00:08.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.554e-05 2024-09-18T01:00:08.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381977+0000", 2024-09-18T01:00:08.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6733000000000001e-05 2024-09-18T01:00:08.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381987+0000", 2024-09-18T01:00:08.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.031e-05 2024-09-18T01:00:08.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475651+0000", 2024-09-18T01:00:08.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093664175000000002 2024-09-18T01:00:08.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475789+0000", 2024-09-18T01:00:08.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000137181 2024-09-18T01:00:08.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475958+0000", 2024-09-18T01:00:08.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016978100000000001 2024-09-18T01:00:08.927 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.927 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.927 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.927 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99195 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 761856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.927 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:08.927 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.440705578999996, 2024-09-18T01:00:08.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961833999999996, 2024-09-18T01:00:08.927 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.927 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99195 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:08.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:08.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:00:08.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382208+0000", 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0469999999999998e-06 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382217+0000", 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5159999999999994e-06 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382224+0000", 2024-09-18T01:00:08.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.64e-06 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.477889+0000", 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095664955999999995 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478012+0000", 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012299899999999999 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478166+0000", 2024-09-18T01:00:08.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015497899999999999 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99196 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 88064~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.440677648999994, 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096312801000000003, 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:08.932 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99196 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:08.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385636+0000", 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034037450000000001 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385642+0000", 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9900000000000002e-06 2024-09-18T01:00:08.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385659+0000", 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6753e-05 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385669+0000", 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9170000000000001e-06 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478225+0000", 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092555514000000005 2024-09-18T01:00:08.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478366+0000", 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014198699999999999 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478545+0000", 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017889499999999999 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:08.936 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:08.937 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T01:00:08.938 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:00:08.952 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:08.941+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:08.978 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6, 7] out_osds: [2, 3, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:00:08.979 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:00:08.979 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-18T01:00:08.979 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:00:08.979 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:00:08.979 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-18T01:00:09.040 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:09.041+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:09.113 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:09.113 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:09.113 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:09.130 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:00:09.140 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T01:00:09.140 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:09.140 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:09.141+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:09.240 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:09.237+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:09.288 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:09.289 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:09.289 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:09.289 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:09.304 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:00:09.340 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:09.337+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:09.431 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 140.02166138199999, 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T01:00:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T01:00:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T01:00:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T01:00:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T01:00:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T01:00:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T01:00:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T01:00:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T01:00:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T01:00:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T01:00:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T01:00:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T01:00:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T01:00:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T01:00:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T01:00:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125622 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 344064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.757526214999999, 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096365904000000002, 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125622 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673347+0000", 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7599999999999997e-07 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673354+0000", 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.486e-06 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673361+0000", 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5520000000000001e-06 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673365+0000", 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7129999999999996e-06 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768923+0000", 2024-09-18T01:00:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095557673999999995 2024-09-18T01:00:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768994+0000", 2024-09-18T01:00:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0847999999999997e-05 2024-09-18T01:00:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769712+0000", 2024-09-18T01:00:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000718055 2024-09-18T01:00:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125623 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.757501800999997, 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096098244999999999, 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125623 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8200000000000001e-07 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673372+0000", 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5519999999999999e-06 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673376+0000", 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.512e-06 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673379+0000", 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1580000000000001e-06 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769200+0000", 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095820946000000004 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769317+0000", 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117003 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769469+0000", 2024-09-18T01:00:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015179200000000001 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125624 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.757489937000003, 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101719451, 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125624 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673383+0000", 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.23e-07 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673384+0000", 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3060000000000001e-06 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673387+0000", 2024-09-18T01:00:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:00:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673390+0000", 2024-09-18T01:00:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:00:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769495+0000", 2024-09-18T01:00:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096104599999999998 2024-09-18T01:00:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769571+0000", 2024-09-18T01:00:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6089999999999998e-05 2024-09-18T01:00:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775102+0000", 2024-09-18T01:00:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055311859999999996 2024-09-18T01:00:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125625 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.757478892999998, 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102242338, 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125625 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673394+0000", 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673395+0000", 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2839999999999999e-06 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673398+0000", 2024-09-18T01:00:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1379999999999999e-06 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673401+0000", 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7149999999999998e-06 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775140+0000", 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101739127 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775212+0000", 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1984999999999996e-05 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775636+0000", 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000423752 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125627 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.757467976000001, 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10248196399999999, 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125627 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673405+0000", 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0699999999999998e-07 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673406+0000", 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2130000000000001e-06 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673409+0000", 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3400000000000002e-06 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673412+0000", 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7360000000000001e-06 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775665+0000", 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102253552 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775734+0000", 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8019e-05 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775886+0000", 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015279700000000001 2024-09-18T01:00:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125628 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.757456877999999, 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.1031562, 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125628 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673416+0000", 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.91e-07 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673418+0000", 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673423+0000", 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6149999999999996e-06 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673428+0000", 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1460000000000004e-06 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776149+0000", 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10272025899999999 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776218+0000", 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9207000000000003e-05 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776572+0000", 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035375599999999999 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125629 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.757437326999998, 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103390994, 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125629 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673438+0000", 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4310000000000001e-06 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673446+0000", 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0290000000000005e-06 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673451+0000", 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3630000000000002e-06 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776600+0000", 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10314894500000001 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776668+0000", 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7963e-05 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776826+0000", 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015784499999999999 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125631 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.757414077, 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.104002186, 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125631 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673459+0000", 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673461+0000", 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.351e-06 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673467+0000", 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3389999999999998e-06 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673472+0000", 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777062+0000", 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103589771 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777130+0000", 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8065999999999995e-05 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777460+0000", 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000330265 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125632 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.757393288999999, 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10711156300000001, 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125632 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6100000000000002e-07 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673482+0000", 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1210000000000001e-06 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673488+0000", 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1659999999999999e-06 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673493+0000", 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.989e-06 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777489+0000", 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10399652600000001 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777565+0000", 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6219999999999999e-05 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780591+0000", 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030251800000000001 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125633 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.757372527000001, 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10735947699999999, 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125633 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673503+0000", 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673509+0000", 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3969999999999999e-06 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673515+0000", 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7640000000000002e-06 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780622+0000", 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.107107533 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780692+0000", 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9735999999999998e-05 2024-09-18T01:00:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780859+0000", 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016721999999999999 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126363 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.719518385999997, 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10567230599999999, 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126363 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711355+0000", 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9999999999999995e-07 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711360+0000", 2024-09-18T01:00:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:00:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711380+0000", 2024-09-18T01:00:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9769e-05 2024-09-18T01:00:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711396+0000", 2024-09-18T01:00:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5947000000000002e-05 2024-09-18T01:00:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817011+0000", 2024-09-18T01:00:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.105615185 2024-09-18T01:00:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817026+0000", 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5430999999999998e-05 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126376 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.708942100999998, 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097651948000000002, 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126376 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721931+0000", 2024-09-18T01:00:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T01:00:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721933+0000", 2024-09-18T01:00:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:00:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721939+0000", 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721944+0000", 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0089999999999998e-06 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817590+0000", 2024-09-18T01:00:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095646089000000004 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.819582+0000", 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019924669999999999 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126427 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.708921568000001, 2024-09-18T01:00:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095671891999999994, 2024-09-18T01:00:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126427 2024-09-18T01:00:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:00:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721954+0000", 2024-09-18T01:00:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4700000000000001e-06 2024-09-18T01:00:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721960+0000", 2024-09-18T01:00:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4629999999999998e-06 2024-09-18T01:00:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721965+0000", 2024-09-18T01:00:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:00:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817614+0000", 2024-09-18T01:00:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095649026999999998 2024-09-18T01:00:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817623+0000", 2024-09-18T01:00:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4780000000000008e-06 2024-09-18T01:00:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126428 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.708899381000002, 2024-09-18T01:00:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095683878999999999, 2024-09-18T01:00:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126428 2024-09-18T01:00:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T01:00:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721976+0000", 2024-09-18T01:00:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4279999999999999e-06 2024-09-18T01:00:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721983+0000", 2024-09-18T01:00:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7499999999999997e-06 2024-09-18T01:00:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721988+0000", 2024-09-18T01:00:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4349999999999996e-06 2024-09-18T01:00:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817644+0000", 2024-09-18T01:00:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095656525000000006 2024-09-18T01:00:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817657+0000", 2024-09-18T01:00:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2312e-05 2024-09-18T01:00:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126429 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.708876341, 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095687929000000005, 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126429 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:00:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721999+0000", 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.649e-06 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722006+0000", 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8299999999999998e-06 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722011+0000", 2024-09-18T01:00:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6970000000000001e-06 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817678+0000", 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095667143999999996 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817684+0000", 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2149999999999998e-06 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126430 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.708854217999999, 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095682481999999999, 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126430 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722019+0000", 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:00:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722021+0000", 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.419e-06 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722027+0000", 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0750000000000003e-06 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722033+0000", 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817695+0000", 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095662607999999996 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817701+0000", 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1359999999999996e-06 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126431 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.708832315000002, 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678416000000002, 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126431 2024-09-18T01:00:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722043+0000", 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722049+0000", 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3219999999999998e-06 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722054+0000", 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817715+0000", 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095660429000000005 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817718+0000", 2024-09-18T01:00:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160401 3.0 3:0686cf5d:::rbd_data.117cf66ad0cc.0000000000000001:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.3596698850000006, 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.89130826799999996, 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160401 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071217+0000", 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4924e-05 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071229+0000", 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.166e-05 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071272+0000", 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2506000000000003e-05 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071294+0000", 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2789000000000002e-05 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071664+0000", 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036966100000000002 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071694+0000", 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0207000000000001e-05 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.955396+0000", 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88370225899999999 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.955864+0000", 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000467997 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.956147+0000", 2024-09-18T01:00:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028220399999999998 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.960924+0000", 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047771760000000002 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962456+0000", 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001532158 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962485+0000", 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8758000000000001e-05 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962511+0000", 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5969000000000002e-05 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160404 3.0 3:048cdd79:::rbd_data.117cf66ad0cc.0000000000000004:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-18T01:00:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.3594928320000008, 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.89253343799999996, 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160404 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071381+0000", 2024-09-18T01:00:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7829999999999999e-06 2024-09-18T01:00:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071390+0000", 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.02e-06 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071417+0000", 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6798999999999999e-05 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071434+0000", 2024-09-18T01:00:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6983e-05 2024-09-18T01:00:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071769+0000", 2024-09-18T01:00:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033536500000000001 2024-09-18T01:00:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071788+0000", 2024-09-18T01:00:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:00:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961054+0000", 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88926583000000003 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961171+0000", 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116468 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961288+0000", 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117759 2024-09-18T01:00:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962832+0000", 2024-09-18T01:00:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001543677 2024-09-18T01:00:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963849+0000", 2024-09-18T01:00:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010164659999999999 2024-09-18T01:00:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963878+0000", 2024-09-18T01:00:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9697e-05 2024-09-18T01:00:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963913+0000", 2024-09-18T01:00:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4628000000000001e-05 2024-09-18T01:00:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.540 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.540 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.540 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.540 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.540 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:09.541 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:00:09.542 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:09.537+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:09.643 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:09.641+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:09.681 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:09.681 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:09.681 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:09.681 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:09.696 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:00:09.743 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:09.741+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:09.829 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:09.830 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:09.830 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:09.830 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:09.830 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:09.844 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:00:09.845 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:09.841+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:09.945 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:09.941+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:09.992 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:09.992 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:09.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:09.992 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:09.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.992 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T01:00:09.992 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:09.992 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 140.902069356, 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T01:00:09.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T01:00:09.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.996 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 140.51117440499999, 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:09.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T01:00:10.000 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141148 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.712760+0000", 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.277923100999999, 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057802683000000001, 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141148 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712760+0000", 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712760+0000", 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712761+0000", 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6729999999999999e-06 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712770+0000", 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6200000000000005e-06 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712791+0000", 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.067e-05 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712807+0000", 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5883e-05 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.764648+0000", 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051841127000000001 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.764855+0000", 2024-09-18T01:00:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020728200000000001 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.765344+0000", 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048933799999999997 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769269+0000", 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0039248900000000003 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770401+0000", 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001132154 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770494+0000", 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2773000000000001e-05 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770562+0000", 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8273000000000002e-05 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141150 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.277843425, 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057991537000000003, 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141150 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712841+0000", 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6330000000000001e-06 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712849+0000", 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1100000000000003e-06 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712873+0000", 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3802999999999999e-05 2024-09-18T01:00:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712889+0000", 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6184e-05 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766071+0000", 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053181379000000001 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766208+0000", 2024-09-18T01:00:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013692399999999999 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766800+0000", 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00059252999999999997 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770601+0000", 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003800997 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770681+0000", 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9975000000000006e-05 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770770+0000", 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9106999999999999e-05 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770831+0000", 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0894999999999999e-05 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141153 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.277281369000001, 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060820664000000003, 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141153 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713403+0000", 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6759999999999999e-06 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713411+0000", 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7670000000000003e-06 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713433+0000", 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1807e-05 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713449+0000", 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5795999999999999e-05 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769325+0000", 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055876605000000003 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769426+0000", 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010125500000000001 2024-09-18T01:00:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769851+0000", 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000424275 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774030+0000", 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041790960000000002 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774050+0000", 2024-09-18T01:00:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.982e-05 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774133+0000", 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3474999999999996e-05 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774222+0000", 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9091999999999993e-05 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141154 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.277191946000002, 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062213932999999999, 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141154 2024-09-18T01:00:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713492+0000", 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.373e-06 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713501+0000", 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6030000000000005e-06 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713538+0000", 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.684e-05 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713555+0000", 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6866000000000001e-05 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771129+0000", 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057574828000000002 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771257+0000", 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012741299999999999 2024-09-18T01:00:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771699+0000", 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044198799999999998 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775574+0000", 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038754229999999998 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775590+0000", 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6010000000000001e-05 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775655+0000", 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4327999999999998e-05 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775705+0000", 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0260999999999999e-05 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141155 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.276626643999997, 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064518352000000001, 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141155 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714059+0000", 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.323e-06 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714070+0000", 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.151e-05 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714099+0000", 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9153e-05 2024-09-18T01:00:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714119+0000", 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9656999999999998e-05 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.772354+0000", 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058234701 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.772479+0000", 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012552700000000001 2024-09-18T01:00:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.773115+0000", 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000635831 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775815+0000", 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026998930000000001 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778455+0000", 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026403279999999999 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778510+0000", 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4611999999999998e-05 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778575+0000", 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4817e-05 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141156 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.273582832999999, 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061742916000000002, 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141156 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717103+0000", 2024-09-18T01:00:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7930000000000002e-06 2024-09-18T01:00:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717119+0000", 2024-09-18T01:00:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676999999999999e-05 2024-09-18T01:00:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717156+0000", 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.701e-05 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717178+0000", 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2806000000000002e-05 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774362+0000", 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057183909999999998 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774527+0000", 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016523899999999999 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774944+0000", 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041637299999999999 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778643+0000", 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003699174 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778737+0000", 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4007999999999998e-05 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778782+0000", 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4886999999999999e-05 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778843+0000", 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1038999999999996e-05 2024-09-18T01:00:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141157 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.273465803999997, 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061753200000000001, 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141157 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717219+0000", 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6330000000000001e-06 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717228+0000", 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9539999999999993e-06 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717251+0000", 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3193e-05 2024-09-18T01:00:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717268+0000", 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.719e-05 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775897+0000", 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058628583999999997 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.776092+0000", 2024-09-18T01:00:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019526600000000001 2024-09-18T01:00:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.776410+0000", 2024-09-18T01:00:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031835700000000001 2024-09-18T01:00:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778719+0000", 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023090910000000001 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778858+0000", 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013843199999999999 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778911+0000", 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2907e-05 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778970+0000", 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9592999999999997e-05 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141158 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.273390986999999, 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064284142000000002, 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141158 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717293+0000", 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3459999999999999e-06 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717322+0000", 2024-09-18T01:00:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8926000000000001e-05 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717345+0000", 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2779999999999999e-05 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717362+0000", 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7079999999999999e-05 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777123+0000", 2024-09-18T01:00:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059760675999999999 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777271+0000", 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014837800000000001 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777835+0000", 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056387600000000005 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781379+0000", 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035443559999999998 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781444+0000", 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4776000000000005e-05 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781515+0000", 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0533999999999999e-05 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781576+0000", 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1414000000000005e-05 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141159 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.273297421999999, 2024-09-18T01:00:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065579987000000006, 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141159 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717387+0000", 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717394+0000", 2024-09-18T01:00:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717413+0000", 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9004e-05 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717428+0000", 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4780000000000001e-05 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779077+0000", 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061649335999999999 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779225+0000", 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000147374 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779649+0000", 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042398799999999998 2024-09-18T01:00:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781615+0000", 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001965732 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782884+0000", 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001269879 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782921+0000", 2024-09-18T01:00:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6927000000000001e-05 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782965+0000", 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4066999999999997e-05 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141160 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.273234299000002, 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.067301246999999995, 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141160 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717450+0000", 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.207e-06 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717456+0000", 2024-09-18T01:00:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.066e-06 2024-09-18T01:00:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717468+0000", 2024-09-18T01:00:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1914e-05 2024-09-18T01:00:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717482+0000", 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3716e-05 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780389+0000", 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062907819000000004 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780502+0000", 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011259100000000001 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780930+0000", 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042795799999999998 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784605+0000", 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036754719999999999 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784631+0000", 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5171999999999999e-05 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784697+0000", 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6506000000000003e-05 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784750+0000", 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2825999999999999e-05 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141165 3.12 3:4e6ddb92:::rbd_data.117c1c3044f5.0000000000000006:27 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.273181076, 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064420416999999994, 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141165 2024-09-18T01:00:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717503+0000", 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2529999999999999e-06 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717510+0000", 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717572+0000", 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2476000000000003e-05 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717588+0000", 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6504e-05 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781669+0000", 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064080210999999998 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781781+0000", 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112137 2024-09-18T01:00:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781922+0000", 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014139900000000001 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141172 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.271998257999996, 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.067304013999999995, 2024-09-18T01:00:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141172 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718687+0000", 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5229999999999998e-06 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718701+0000", 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3397000000000001e-05 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718728+0000", 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7350000000000001e-05 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718748+0000", 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0205999999999999e-05 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781972+0000", 2024-09-18T01:00:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.063223835000000006 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782067+0000", 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5206999999999998e-05 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782420+0000", 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035256799999999999 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784838+0000", 2024-09-18T01:00:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002418023 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785893+0000", 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010552440000000001 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785945+0000", 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1706999999999998e-05 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785989+0000", 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3954e-05 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141175 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.269980703000002, 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.066834381999999998, 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141175 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720705+0000", 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.79e-06 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720720+0000", 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4897e-05 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720754+0000", 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4323000000000003e-05 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720776+0000", 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1535999999999998e-05 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783058+0000", 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062282205 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783203+0000", 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000144816 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783831+0000", 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062834200000000005 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787404+0000", 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035724429999999998 2024-09-18T01:00:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787419+0000", 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5783000000000001e-05 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787468+0000", 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9107999999999998e-05 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787537+0000", 2024-09-18T01:00:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8139000000000006e-05 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141176 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.269870767, 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.068349919999999995, 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141176 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720814+0000", 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.457e-06 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720822+0000", 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5969999999999992e-06 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720844+0000", 2024-09-18T01:00:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1965999999999999e-05 2024-09-18T01:00:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720860+0000", 2024-09-18T01:00:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6243e-05 2024-09-18T01:00:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784925+0000", 2024-09-18T01:00:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064065009000000006 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785025+0000", 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9854999999999999e-05 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785407+0000", 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038140800000000002 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787604+0000", 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021976389999999999 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789028+0000", 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001423304 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789103+0000", 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5284999999999995e-05 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789162+0000", 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9157000000000003e-05 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141177 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.269800914000001, 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.069464445999999999, 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141177 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720883+0000", 2024-09-18T01:00:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.37e-06 2024-09-18T01:00:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720891+0000", 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8299999999999996e-06 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720916+0000", 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5216e-05 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720932+0000", 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676999999999999e-05 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786055+0000", 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065122920000000001 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786179+0000", 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000124153 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786832+0000", 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065315700000000002 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790206+0000", 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003374008 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790230+0000", 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3963000000000001e-05 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790291+0000", 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0747e-05 2024-09-18T01:00:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790346+0000", 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5405000000000002e-05 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141179 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.268815324999998, 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.070779396999999994, 2024-09-18T01:00:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141179 2024-09-18T01:00:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721870+0000", 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5739999999999998e-06 2024-09-18T01:00:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721885+0000", 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4443e-05 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721917+0000", 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2122999999999997e-05 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721938+0000", 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1583e-05 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787900+0000", 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065962144 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.788009+0000", 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010851799999999999 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.788372+0000", 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036301900000000002 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.791319+0000", 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029471609999999998 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792566+0000", 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001247155 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792615+0000", 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792647+0000", 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1699000000000003e-05 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141180 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.268194553000001, 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.070290325000000001, 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141180 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722491+0000", 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2024-09-18T01:00:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722503+0000", 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2653000000000001e-05 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722532+0000", 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9166000000000002e-05 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722582+0000", 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:00:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789227+0000", 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.066644492999999999 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789286+0000", 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8959999999999998e-05 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789638+0000", 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035211500000000001 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792662+0000", 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030245369999999999 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792699+0000", 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6810999999999997e-05 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792749+0000", 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9885000000000001e-05 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792779+0000", 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9794e-05 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:150977 3.b 3:d4ab7022:::rbd_data.117c741c3c2b.000000000000000c:2a [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-18T01:00:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.390829824000001, 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064163676000000003, 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 150977 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599854+0000", 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599860+0000", 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5330000000000002e-06 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599877+0000", 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7056000000000001e-05 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599891+0000", 2024-09-18T01:00:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4569999999999999e-05 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.652536+0000", 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052644519000000001 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.652629+0000", 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3255000000000001e-05 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.664017+0000", 2024-09-18T01:00:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011387516 2024-09-18T01:00:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:10.107 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:10.107 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:00:10.109 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:10.105+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:10.109 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:00:10.209 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:10.205+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:10.246 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:10.247 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:10.247 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:10.247 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:10.262 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:00:10.275 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:10.275 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:10.299 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:10.309 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:10.309+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:10.317 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:10.336 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:10.336 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:10.356 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:10.356 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:10.377 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:10.377 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:10.393 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:10.393 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:10.393 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:10.393 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:10.394 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:10.399 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:10.399 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:10.409 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:00:10.422 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:10.423 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:10.452 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:10.453+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:10.511 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:10.509+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:10.593 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:10.594 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:10.613 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:10.613+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:10.613 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:00:10.713 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:10.713+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:10.749 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:10.750 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:10.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:00:10.814 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:10.813+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:10.883 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:10.883 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:10.899 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:00:10.916 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:10.913+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:10.954 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:00:11.030 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:11.029+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:11.069 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:11.069 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:11.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:00:11.117 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:11.117+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:11.218 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:11.217+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:11.244 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:11.244 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:11.258 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:00:11.319 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:11.316+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:11.420 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:11.420+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:11.421 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:11.421 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:11.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:00:11.521 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:11.520+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:11.562 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:11.562 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:11.576 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:00:11.621 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:11.620+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:11.704 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:11.704 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:11.719 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:00:11.722 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:11.720+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:11.823 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:11.820+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:11.846 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:11.846 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:11.860 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:00:11.917 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T01:00:11.925 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:11.924+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:11.988 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:11.989 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:12.003 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:00:12.025 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:12.024+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:12.124 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:12.124+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:12.125 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:12.125 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:12.141 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:00:12.226 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:12.225+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:12.272 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:12.273 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:12.287 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:00:12.330 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:12.329+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:12.428 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:12.425+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:12.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:00:12.455 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:12.455 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:12.471 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:00:12.529 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:12.528+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:12.595 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:12.595 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:12.603 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:12.603 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:12.611 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:12.611 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:12.617 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:00:12.628 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:12.628 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:12.629 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:12.625+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:12.645 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:12.645 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:12.664 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:12.664 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:12.683 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:12.683 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:12.701 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:12.701 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:12.722 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:12.722 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:12.730 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:12.728+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:12.786 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:12.786 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:12.786 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:12.800 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:00:12.831 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:12.829+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:12.933 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:12.929+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:12.980 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:12.980 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:12.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:12.980 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:12.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:9366 3.2 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 144.20308312500001, 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.207652744, 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 9366 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:12.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774863+0000", 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.774865+0000", 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846842+0000", 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.071977088999999994 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846895+0000", 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2553000000000003e-05 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.846915+0000", 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0326000000000002e-05 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847086+0000", 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171593 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.847514+0000", 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042765300000000001 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 2", 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.850828+0000", 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033139620000000002 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.956760+0000", 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10593190600000001 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982320+0000", 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025560367000000001 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982434+0000", 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011376599999999999 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.982516+0000", 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1686000000000004e-05 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87406 3.19 3:9d734bda:::rbd_data.117c8934add3.000000000000007c:head [stat,delete] snapc 1f=[1f] RETRY=8 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.555160002999997, 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097024104999999999, 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:12.986 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87406 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.422786+0000", 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:12.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425532+0000", 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027458309999999998 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425537+0000", 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2839999999999997e-06 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425551+0000", 2024-09-18T01:00:12.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3682999999999999e-05 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425559+0000", 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5329999999999997e-06 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515035+0000", 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.089476411000000006 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515168+0000", 2024-09-18T01:00:12.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013309699999999999 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.515475+0000", 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030662199999999999 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519553+0000", 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040781259999999996 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519665+0000", 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112669 2024-09-18T01:00:12.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519748+0000", 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2270999999999999e-05 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.519810+0000", 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2577999999999996e-05 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.991 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:87414 3.19 3:9c145b25:::rbd_data.117c8934add3.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1f=[1f] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 84.552336464000007, 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095886693999999995, 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 87414 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:12.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:12.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:12.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:12.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:12.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:12.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425610+0000", 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425618+0000", 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9810000000000003e-06 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425625+0000", 2024-09-18T01:00:12.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0870000000000001e-06 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.425630+0000", 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.262e-06 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518417+0000", 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092787816999999995 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:12.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518593+0000", 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017569100000000001 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.518939+0000", 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034572299999999998 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521320+0000", 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023809619999999999 2024-09-18T01:00:12.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521343+0000", 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3317000000000001e-05 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521422+0000", 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480999999999996e-05 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:48.521496+0000", 2024-09-18T01:00:12.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4909999999999999e-05 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99159 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 393216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.608680113999995, 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096008205999999999, 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.000 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.001 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.001 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99159 2024-09-18T01:00:13.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.001 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:13.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369266+0000", 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369409+0000", 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014342599999999999 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369415+0000", 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.31e-06 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369426+0000", 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1677e-05 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369435+0000", 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.464949+0000", 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095513718999999997 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465099+0000", 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014984300000000001 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465274+0000", 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017554499999999999 2024-09-18T01:00:13.004 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99160 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2336768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.608495544999997, 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096064584999999994, 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.005 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99160 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369451+0000", 2024-09-18T01:00:13.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369564+0000", 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113309 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369569+0000", 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5269999999999998e-06 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369582+0000", 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2952999999999999e-05 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369592+0000", 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0939999999999997e-06 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465328+0000", 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095736285000000004 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465403+0000", 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5498000000000001e-05 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465515+0000", 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111919 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.009 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99161 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 413696~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.608337915999996, 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096302270999999995, 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99161 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.369608+0000", 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373792+0000", 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004184063 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373801+0000", 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3729999999999993e-06 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373818+0000", 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7280000000000001e-05 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.373829+0000", 2024-09-18T01:00:13.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0540000000000001e-05 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465596+0000", 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.091767910999999994 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465740+0000", 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143683 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.465911+0000", 2024-09-18T01:00:13.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000170421 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99172 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2459648~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.603742573000005, 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095842035000000006, 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.014 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99172 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:13.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374204+0000", 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9299999999999997e-07 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374207+0000", 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8899999999999999e-06 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374216+0000", 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3669999999999997e-06 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374223+0000", 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9759999999999998e-06 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469763+0000", 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095540307000000005 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.469894+0000", 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013135600000000001 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470046+0000", 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015144600000000001 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.018 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99173 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2480128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.603715012999999, 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096195265000000002, 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99173 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374231+0000", 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374232+0000", 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4600000000000004e-07 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374235+0000", 2024-09-18T01:00:13.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8569999999999999e-06 2024-09-18T01:00:13.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374243+0000", 2024-09-18T01:00:13.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5930000000000006e-06 2024-09-18T01:00:13.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374250+0000", 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9330000000000004e-06 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470113+0000", 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095862952000000001 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470259+0000", 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014534300000000001 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470426+0000", 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016784100000000001 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99174 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2500608~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.603686706999994, 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096540567999999993, 2024-09-18T01:00:13.023 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99174 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374259+0000", 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374260+0000", 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2600000000000002e-07 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374263+0000", 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8770000000000002e-06 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374272+0000", 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6070000000000008e-06 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374279+0000", 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8430000000000004e-06 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470492+0000", 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096213513000000001 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470632+0000", 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013954899999999999 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470800+0000", 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016845299999999999 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.027 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99175 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 536576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.603658811000003, 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096867154999999996, 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99175 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374287+0000", 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374288+0000", 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9999999999999997e-07 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374291+0000", 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.92e-06 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374300+0000", 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5939999999999994e-06 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374307+0000", 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9829999999999999e-06 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470858+0000", 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096551085999999994 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.470990+0000", 2024-09-18T01:00:13.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013223900000000001 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471155+0000", 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016463299999999999 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99176 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2521088~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:13.032 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.603630860999999, 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097204272999999994, 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99176 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374315+0000", 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374316+0000", 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1999999999999999e-07 2024-09-18T01:00:13.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374336+0000", 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9879999999999999e-05 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374346+0000", 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0353999999999999e-05 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374354+0000", 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7130000000000005e-06 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471210+0000", 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096856271999999993 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471350+0000", 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013940900000000001 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471520+0000", 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000169925 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99177 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 557056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.603581997999996, 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097507736999999997, 2024-09-18T01:00:13.037 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99177 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:13.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.374364+0000", 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377919+0000", 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035553070000000002 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377927+0000", 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4569999999999999e-06 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377943+0000", 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5563e-05 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.377953+0000", 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0413e-05 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471578+0000", 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093625036999999994 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471710+0000", 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013170900000000001 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.471872+0000", 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162251 2024-09-18T01:00:13.041 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99183 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 618496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.599793962000007, 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095979540000000002, 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99183 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378152+0000", 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378153+0000", 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.23e-07 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378156+0000", 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9340000000000002e-06 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378164+0000", 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4630000000000001e-06 2024-09-18T01:00:13.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378173+0000", 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8130000000000002e-06 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473851+0000", 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678257000000003 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.473982+0000", 2024-09-18T01:00:13.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000130875 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474132+0000", 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014947499999999999 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99184 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 2603008~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.599737419999997, 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096289871999999999, 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.046 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99184 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378209+0000", 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378210+0000", 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2999999999999999e-07 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378213+0000", 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4970000000000001e-06 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378223+0000", 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4700000000000008e-06 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378230+0000", 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5830000000000001e-06 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474192+0000", 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961742000000003 2024-09-18T01:00:13.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474324+0000", 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000132099 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474499+0000", 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174551 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99185 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 638976~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.599702992999994, 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096618187999999994, 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.051 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99185 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378243+0000", 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378244+0000", 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5700000000000001e-07 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378248+0000", 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5130000000000001e-06 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378256+0000", 2024-09-18T01:00:13.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8429999999999999e-06 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378263+0000", 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9870000000000002e-06 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474554+0000", 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096290693999999996 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474692+0000", 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013761900000000001 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474861+0000", 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016967499999999999 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.055 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99186 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 659456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.599673443, 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096958847000000001, 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99186 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378273+0000", 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1699999999999997e-07 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378277+0000", 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0800000000000002e-06 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378285+0000", 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5699999999999993e-06 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378292+0000", 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9600000000000003e-06 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.474924+0000", 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096632207999999997 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475067+0000", 2024-09-18T01:00:13.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000143175 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475232+0000", 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016413699999999999 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99187 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 679936~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.060 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.599645170000002, 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097293710000000005, 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99187 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378301+0000", 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378302+0000", 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378305+0000", 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.066e-06 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378315+0000", 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0443e-05 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378323+0000", 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3239999999999997e-06 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475291+0000", 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096968726000000005 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475430+0000", 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013847499999999999 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475595+0000", 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016497899999999999 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.064 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99188 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 700416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.599614333999995, 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097626477000000003, 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.065 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99188 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.378332+0000", 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381945+0000", 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036127569999999999 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381960+0000", 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.554e-05 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381977+0000", 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6733000000000001e-05 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.381987+0000", 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.031e-05 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475651+0000", 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.093664175000000002 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475789+0000", 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000137181 2024-09-18T01:00:13.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.475958+0000", 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016978100000000001 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99195 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 761856~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.595741528999994, 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095961833999999996, 2024-09-18T01:00:13.069 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99195 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382205+0000", 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9699999999999995e-07 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382208+0000", 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0469999999999998e-06 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382217+0000", 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5159999999999994e-06 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382224+0000", 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.64e-06 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.477889+0000", 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095664955999999995 2024-09-18T01:00:13.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478012+0000", 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012299899999999999 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478166+0000", 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015497899999999999 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.073 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:99196 3.19 3:9da12e99:::rbd_data.117c304cc916.0000000000000000:1e [read 88064~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.595713599000007, 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096312801000000003, 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 99196 2024-09-18T01:00:13.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.382233+0000", 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385636+0000", 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034037450000000001 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385642+0000", 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9900000000000002e-06 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385659+0000", 2024-09-18T01:00:13.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6753e-05 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.385669+0000", 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9170000000000001e-06 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478225+0000", 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.092555514000000005 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478366+0000", 2024-09-18T01:00:13.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014198699999999999 2024-09-18T01:00:13.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:59.478545+0000", 2024-09-18T01:00:13.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017889499999999999 2024-09-18T01:00:13.078 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.078 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.078 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.078 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.078 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.079 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:13.080 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:00:13.081 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:13.077+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:13.181 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:13.177+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:13.219 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:13.219 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:13.219 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:13.219 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:13.234 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:00:13.281 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:13.280+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:13.356 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:13.356 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:13.356 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:13.369 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:00:13.382 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:13.381+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:13.483 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:13.480+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:13.540 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 144.129588541, 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:13.541 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:13.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T01:00:13.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125622 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 344064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.865453373999998, 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096365904000000002, 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.547 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125622 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673347+0000", 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7599999999999997e-07 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673354+0000", 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.486e-06 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673361+0000", 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5520000000000001e-06 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673365+0000", 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7129999999999996e-06 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768923+0000", 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095557673999999995 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768994+0000", 2024-09-18T01:00:13.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0847999999999997e-05 2024-09-18T01:00:13.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769712+0000", 2024-09-18T01:00:13.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000718055 2024-09-18T01:00:13.551 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.551 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125623 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.865428960000003, 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096098244999999999, 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.552 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125623 2024-09-18T01:00:13.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.553 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:13.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8200000000000001e-07 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673372+0000", 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5519999999999999e-06 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673376+0000", 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.512e-06 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673379+0000", 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1580000000000001e-06 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769200+0000", 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095820946000000004 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769317+0000", 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117003 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769469+0000", 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015179200000000001 2024-09-18T01:00:13.556 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125624 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.865417096000002, 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101719451, 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.557 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125624 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:13.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673383+0000", 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.23e-07 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673384+0000", 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3060000000000001e-06 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673387+0000", 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673390+0000", 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769495+0000", 2024-09-18T01:00:13.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096104599999999998 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769571+0000", 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6089999999999998e-05 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775102+0000", 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055311859999999996 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.561 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125625 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.865406051999997, 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102242338, 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125625 2024-09-18T01:00:13.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673394+0000", 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673395+0000", 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2839999999999999e-06 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673398+0000", 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1379999999999999e-06 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673401+0000", 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7149999999999998e-06 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775140+0000", 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101739127 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775212+0000", 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1984999999999996e-05 2024-09-18T01:00:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775636+0000", 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000423752 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125627 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.865395135, 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10248196399999999, 2024-09-18T01:00:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125627 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673405+0000", 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0699999999999998e-07 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673406+0000", 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2130000000000001e-06 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673409+0000", 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3400000000000002e-06 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673412+0000", 2024-09-18T01:00:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7360000000000001e-06 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775665+0000", 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102253552 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775734+0000", 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8019e-05 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775886+0000", 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015279700000000001 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125628 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.865384036999998, 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.1031562, 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125628 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673416+0000", 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.91e-07 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673418+0000", 2024-09-18T01:00:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673423+0000", 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6149999999999996e-06 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673428+0000", 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1460000000000004e-06 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776149+0000", 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10272025899999999 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776218+0000", 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9207000000000003e-05 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776572+0000", 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035375599999999999 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125629 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.865364486000004, 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103390994, 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125629 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673438+0000", 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4310000000000001e-06 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673446+0000", 2024-09-18T01:00:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0290000000000005e-06 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673451+0000", 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3630000000000002e-06 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776600+0000", 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10314894500000001 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776668+0000", 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7963e-05 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776826+0000", 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015784499999999999 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125631 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.865341236000006, 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.104002186, 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125631 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673459+0000", 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673461+0000", 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.351e-06 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673467+0000", 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3389999999999998e-06 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673472+0000", 2024-09-18T01:00:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777062+0000", 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103589771 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777130+0000", 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8065999999999995e-05 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777460+0000", 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000330265 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125632 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.865320448000006, 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10711156300000001, 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125632 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6100000000000002e-07 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673482+0000", 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1210000000000001e-06 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673488+0000", 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1659999999999999e-06 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673493+0000", 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.989e-06 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777489+0000", 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10399652600000001 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777565+0000", 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6219999999999999e-05 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780591+0000", 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030251800000000001 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125633 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.865299686, 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10735947699999999, 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125633 2024-09-18T01:00:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673503+0000", 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673509+0000", 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3969999999999999e-06 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673515+0000", 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7640000000000002e-06 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780622+0000", 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.107107533 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780692+0000", 2024-09-18T01:00:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9735999999999998e-05 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780859+0000", 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016721999999999999 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126363 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.827445545000003, 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10567230599999999, 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126363 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711355+0000", 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9999999999999995e-07 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711360+0000", 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711380+0000", 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9769e-05 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711396+0000", 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5947000000000002e-05 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817011+0000", 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.105615185 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817026+0000", 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5430999999999998e-05 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126376 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.816869260000004, 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097651948000000002, 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126376 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721931+0000", 2024-09-18T01:00:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721933+0000", 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721939+0000", 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721944+0000", 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0089999999999998e-06 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817590+0000", 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095646089000000004 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.819582+0000", 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019924669999999999 2024-09-18T01:00:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126427 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.816848727000007, 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095671891999999994, 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126427 2024-09-18T01:00:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721954+0000", 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4700000000000001e-06 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721960+0000", 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4629999999999998e-06 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721965+0000", 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:00:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817614+0000", 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095649026999999998 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817623+0000", 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4780000000000008e-06 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126428 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.816826539999994, 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095683878999999999, 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126428 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721976+0000", 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4279999999999999e-06 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721983+0000", 2024-09-18T01:00:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7499999999999997e-06 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721988+0000", 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4349999999999996e-06 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817644+0000", 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095656525000000006 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817657+0000", 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2312e-05 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126429 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.816803500000006, 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095687929000000005, 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126429 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721999+0000", 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.649e-06 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722006+0000", 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8299999999999998e-06 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722011+0000", 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6970000000000001e-06 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817678+0000", 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095667143999999996 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817684+0000", 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2149999999999998e-06 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126430 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.816781376999998, 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095682481999999999, 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126430 2024-09-18T01:00:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722019+0000", 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722021+0000", 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.419e-06 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722027+0000", 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0750000000000003e-06 2024-09-18T01:00:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722033+0000", 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817695+0000", 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095662607999999996 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817701+0000", 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1359999999999996e-06 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126431 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.816759473999994, 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678416000000002, 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126431 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722043+0000", 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722049+0000", 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3219999999999998e-06 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722054+0000", 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817715+0000", 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095660429000000005 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817718+0000", 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160401 3.0 3:0686cf5d:::rbd_data.117cf66ad0cc.0000000000000001:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.467597044, 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.89130826799999996, 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160401 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071217+0000", 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4924e-05 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071229+0000", 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.166e-05 2024-09-18T01:00:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071272+0000", 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2506000000000003e-05 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071294+0000", 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2789000000000002e-05 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071664+0000", 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036966100000000002 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071694+0000", 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0207000000000001e-05 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.955396+0000", 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88370225899999999 2024-09-18T01:00:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.955864+0000", 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000467997 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.956147+0000", 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028220399999999998 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.960924+0000", 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047771760000000002 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962456+0000", 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001532158 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962485+0000", 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8758000000000001e-05 2024-09-18T01:00:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962511+0000", 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5969000000000002e-05 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160404 3.0 3:048cdd79:::rbd_data.117cf66ad0cc.0000000000000004:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.467419991, 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.89253343799999996, 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160404 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071381+0000", 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7829999999999999e-06 2024-09-18T01:00:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071390+0000", 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.02e-06 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071417+0000", 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6798999999999999e-05 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071434+0000", 2024-09-18T01:00:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6983e-05 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071769+0000", 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033536500000000001 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071788+0000", 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961054+0000", 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88926583000000003 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961171+0000", 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116468 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961288+0000", 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117759 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962832+0000", 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001543677 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963849+0000", 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010164659999999999 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963878+0000", 2024-09-18T01:00:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9697e-05 2024-09-18T01:00:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963913+0000", 2024-09-18T01:00:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4628000000000001e-05 2024-09-18T01:00:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:13.642 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:13.644 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:00:13.644 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:13.644+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:13.745 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:13.744+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:13.814 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:13.814 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:13.814 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:13.814 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:13.828 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:00:13.845 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:13.845+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:13.945 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:00:13.947 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:13.945+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:14.012 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:14.012 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:14.012 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:14.013 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:14.013 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:14.027 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:00:14.047 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:14.045+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:14.147 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:14.144+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:14.158 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6, 7] out_osds: [2, 3, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:00:14.158 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:00:14.158 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-18T01:00:14.158 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-18T01:00:14.160 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:14.160 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:14.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:14.160 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:14.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.160 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T01:00:14.160 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:14.160 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 145.069863553, 2024-09-18T01:00:14.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T01:00:14.160 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.160 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T01:00:14.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 144.678968602, 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T01:00:14.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141148 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.712760+0000", 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.445717297999998, 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057802683000000001, 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.169 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141148 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712760+0000", 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712760+0000", 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712761+0000", 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6729999999999999e-06 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712770+0000", 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6200000000000005e-06 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712791+0000", 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.067e-05 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712807+0000", 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5883e-05 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.764648+0000", 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051841127000000001 2024-09-18T01:00:14.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.764855+0000", 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020728200000000001 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.765344+0000", 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048933799999999997 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769269+0000", 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0039248900000000003 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770401+0000", 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001132154 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770494+0000", 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2773000000000001e-05 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770562+0000", 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8273000000000002e-05 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141150 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.445637622, 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057991537000000003, 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.175 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141150 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712841+0000", 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6330000000000001e-06 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712849+0000", 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1100000000000003e-06 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712873+0000", 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3802999999999999e-05 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712889+0000", 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6184e-05 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766071+0000", 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053181379000000001 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766208+0000", 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013692399999999999 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766800+0000", 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00059252999999999997 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770601+0000", 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003800997 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770681+0000", 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9975000000000006e-05 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770770+0000", 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9106999999999999e-05 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770831+0000", 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0894999999999999e-05 2024-09-18T01:00:14.180 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.181 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.181 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.181 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141153 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.181 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:14.181 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.445075566, 2024-09-18T01:00:14.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060820664000000003, 2024-09-18T01:00:14.181 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.181 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141153 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713403+0000", 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6759999999999999e-06 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713411+0000", 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7670000000000003e-06 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713433+0000", 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1807e-05 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713449+0000", 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5795999999999999e-05 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769325+0000", 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055876605000000003 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769426+0000", 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010125500000000001 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769851+0000", 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000424275 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774030+0000", 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041790960000000002 2024-09-18T01:00:14.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774050+0000", 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.982e-05 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774133+0000", 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3474999999999996e-05 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774222+0000", 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9091999999999993e-05 2024-09-18T01:00:14.186 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141154 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.444986143000001, 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062213932999999999, 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.187 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141154 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713492+0000", 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.373e-06 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713501+0000", 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6030000000000005e-06 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713538+0000", 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.684e-05 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713555+0000", 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6866000000000001e-05 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771129+0000", 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057574828000000002 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771257+0000", 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012741299999999999 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771699+0000", 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044198799999999998 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775574+0000", 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038754229999999998 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775590+0000", 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6010000000000001e-05 2024-09-18T01:00:14.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775655+0000", 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4327999999999998e-05 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775705+0000", 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0260999999999999e-05 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141155 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.444420841000003, 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064518352000000001, 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141155 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714059+0000", 2024-09-18T01:00:14.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.323e-06 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714070+0000", 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.151e-05 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714099+0000", 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9153e-05 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714119+0000", 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9656999999999998e-05 2024-09-18T01:00:14.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.772354+0000", 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058234701 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.772479+0000", 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012552700000000001 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.773115+0000", 2024-09-18T01:00:14.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000635831 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775815+0000", 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026998930000000001 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778455+0000", 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026403279999999999 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778510+0000", 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4611999999999998e-05 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778575+0000", 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4817e-05 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141156 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.441377029999998, 2024-09-18T01:00:14.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061742916000000002, 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141156 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717103+0000", 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7930000000000002e-06 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717119+0000", 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676999999999999e-05 2024-09-18T01:00:14.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717156+0000", 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.701e-05 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717178+0000", 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2806000000000002e-05 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774362+0000", 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057183909999999998 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774527+0000", 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016523899999999999 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774944+0000", 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041637299999999999 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778643+0000", 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003699174 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778737+0000", 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4007999999999998e-05 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778782+0000", 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4886999999999999e-05 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778843+0000", 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1038999999999996e-05 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141157 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.441260001000003, 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061753200000000001, 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.204 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141157 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:14.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717219+0000", 2024-09-18T01:00:14.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6330000000000001e-06 2024-09-18T01:00:14.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717228+0000", 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9539999999999993e-06 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717251+0000", 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3193e-05 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717268+0000", 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.719e-05 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775897+0000", 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058628583999999997 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.776092+0000", 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019526600000000001 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.776410+0000", 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031835700000000001 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778719+0000", 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023090910000000001 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778858+0000", 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013843199999999999 2024-09-18T01:00:14.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778911+0000", 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2907e-05 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778970+0000", 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9592999999999997e-05 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141158 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.441185183999998, 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064284142000000002, 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141158 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717293+0000", 2024-09-18T01:00:14.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3459999999999999e-06 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717322+0000", 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8926000000000001e-05 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717345+0000", 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2779999999999999e-05 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717362+0000", 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7079999999999999e-05 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777123+0000", 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059760675999999999 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777271+0000", 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014837800000000001 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777835+0000", 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056387600000000005 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781379+0000", 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035443559999999998 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781444+0000", 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4776000000000005e-05 2024-09-18T01:00:14.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781515+0000", 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0533999999999999e-05 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781576+0000", 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1414000000000005e-05 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141159 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.441091618999998, 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065579987000000006, 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141159 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.217 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717387+0000", 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T01:00:14.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717394+0000", 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717413+0000", 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9004e-05 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717428+0000", 2024-09-18T01:00:14.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4780000000000001e-05 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779077+0000", 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061649335999999999 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779225+0000", 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000147374 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779649+0000", 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042398799999999998 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781615+0000", 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001965732 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782884+0000", 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001269879 2024-09-18T01:00:14.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782921+0000", 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6927000000000001e-05 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782965+0000", 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4066999999999997e-05 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.222 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141160 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.441028496000001, 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.067301246999999995, 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141160 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717450+0000", 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.207e-06 2024-09-18T01:00:14.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717456+0000", 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.066e-06 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717468+0000", 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1914e-05 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717482+0000", 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3716e-05 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780389+0000", 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062907819000000004 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780502+0000", 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011259100000000001 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780930+0000", 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042795799999999998 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784605+0000", 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036754719999999999 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784631+0000", 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5171999999999999e-05 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784697+0000", 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6506000000000003e-05 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784750+0000", 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2825999999999999e-05 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141165 3.12 3:4e6ddb92:::rbd_data.117c1c3044f5.0000000000000006:27 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.228 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.440975272999999, 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064420416999999994, 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141165 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717503+0000", 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2529999999999999e-06 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717510+0000", 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717572+0000", 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2476000000000003e-05 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717588+0000", 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6504e-05 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781669+0000", 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064080210999999998 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781781+0000", 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112137 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781922+0000", 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014139900000000001 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.232 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141172 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.439792455000003, 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.067304013999999995, 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.233 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141172 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718687+0000", 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5229999999999998e-06 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718701+0000", 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3397000000000001e-05 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718728+0000", 2024-09-18T01:00:14.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7350000000000001e-05 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718748+0000", 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0205999999999999e-05 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781972+0000", 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.063223835000000006 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782067+0000", 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5206999999999998e-05 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782420+0000", 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035256799999999999 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784838+0000", 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002418023 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785893+0000", 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010552440000000001 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785945+0000", 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1706999999999998e-05 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785989+0000", 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3954e-05 2024-09-18T01:00:14.238 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141175 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.437774900000001, 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.066834381999999998, 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.239 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141175 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720705+0000", 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.79e-06 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720720+0000", 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4897e-05 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720754+0000", 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4323000000000003e-05 2024-09-18T01:00:14.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720776+0000", 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1535999999999998e-05 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783058+0000", 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062282205 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783203+0000", 2024-09-18T01:00:14.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000144816 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783831+0000", 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062834200000000005 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787404+0000", 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035724429999999998 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787419+0000", 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5783000000000001e-05 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787468+0000", 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9107999999999998e-05 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787537+0000", 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8139000000000006e-05 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.244 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141176 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.437664964, 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.068349919999999995, 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.245 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141176 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720814+0000", 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.457e-06 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720822+0000", 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5969999999999992e-06 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720844+0000", 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1965999999999999e-05 2024-09-18T01:00:14.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720860+0000", 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6243e-05 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784925+0000", 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064065009000000006 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785025+0000", 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9854999999999999e-05 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785407+0000", 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038140800000000002 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787604+0000", 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021976389999999999 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789028+0000", 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001423304 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789103+0000", 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5284999999999995e-05 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789162+0000", 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9157000000000003e-05 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.250 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141177 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.437595111, 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.069464445999999999, 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.251 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141177 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720883+0000", 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.37e-06 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720891+0000", 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8299999999999996e-06 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720916+0000", 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5216e-05 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720932+0000", 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676999999999999e-05 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786055+0000", 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065122920000000001 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786179+0000", 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000124153 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786832+0000", 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065315700000000002 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790206+0000", 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003374008 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790230+0000", 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3963000000000001e-05 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790291+0000", 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0747e-05 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790346+0000", 2024-09-18T01:00:14.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5405000000000002e-05 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141179 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.436609521999998, 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.070779396999999994, 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.257 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141179 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:14.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721870+0000", 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5739999999999998e-06 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721885+0000", 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4443e-05 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721917+0000", 2024-09-18T01:00:14.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2122999999999997e-05 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721938+0000", 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1583e-05 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787900+0000", 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065962144 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.788009+0000", 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010851799999999999 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.788372+0000", 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036301900000000002 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.791319+0000", 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029471609999999998 2024-09-18T01:00:14.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792566+0000", 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001247155 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792615+0000", 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792647+0000", 2024-09-18T01:00:14.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1699000000000003e-05 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141180 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.43598875, 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.070290325000000001, 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.263 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141180 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722491+0000", 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722503+0000", 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2653000000000001e-05 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722532+0000", 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9166000000000002e-05 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722582+0000", 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789227+0000", 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.066644492999999999 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789286+0000", 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8959999999999998e-05 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789638+0000", 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035211500000000001 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792662+0000", 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030245369999999999 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792699+0000", 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6810999999999997e-05 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792749+0000", 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9885000000000001e-05 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792779+0000", 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9794e-05 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.268 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:150977 3.b 3:d4ab7022:::rbd_data.117c741c3c2b.000000000000000c:2a [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.558624021, 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064163676000000003, 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.269 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 150977 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599854+0000", 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599860+0000", 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5330000000000002e-06 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599877+0000", 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7056000000000001e-05 2024-09-18T01:00:14.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599891+0000", 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4569999999999999e-05 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.652536+0000", 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052644519000000001 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.652629+0000", 2024-09-18T01:00:14.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3255000000000001e-05 2024-09-18T01:00:14.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.664017+0000", 2024-09-18T01:00:14.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011387516 2024-09-18T01:00:14.273 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.273 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.273 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.273 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.273 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.273 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:14.275 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:00:14.275 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:14.272+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:14.376 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:14.373+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:14.444 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:14.445 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:14.445 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:14.445 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:14.459 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:00:14.508 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:14.508+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:14.577 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:14.573+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:14.577 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:14.578 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:14.578 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:14.578 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:14.578 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:14.584 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-18T01:00:14.581+0000 7fdb35eeb640 -1 mon.a@0(leader).osd e156 definitely_dead 0 2024-09-18T01:00:14.592 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:00:14.680 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:14.680+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:14.741 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:00:14.763 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:14.763 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:14.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:14.763 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:14.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.763 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:14.763 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:14.763 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 149.757752635, 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T01:00:14.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T01:00:14.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T01:00:14.767 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 149.757694378, 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.768 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T01:00:14.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T01:00:14.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T01:00:14.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T01:00:14.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T01:00:14.772 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.772 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.772 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.772 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 149.75763593900001, 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T01:00:14.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 149.75757472199999, 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T01:00:14.777 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T01:00:14.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T01:00:14.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.781 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 149.757484307, 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.782 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T01:00:14.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T01:00:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 149.75741337400001, 2024-09-18T01:00:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T01:00:14.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 149.757353768, 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.791 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:14.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.795 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 149.757294398, 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T01:00:14.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T01:00:14.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T01:00:14.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 149.757232609, 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.800 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T01:00:14.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 149.757099786, 2024-09-18T01:00:14.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T01:00:14.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T01:00:14.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T01:00:14.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T01:00:14.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T01:00:14.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 146.20188230900001, 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.809 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T01:00:14.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 146.201382185, 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.815 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:14.819 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 146.20135167699999, 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:14.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T01:00:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 146.20132701200001, 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:00:14.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T01:00:14.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 146.20129847999999, 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.828 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T01:00:14.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:14.832 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 146.20096518899999, 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.836 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 146.200923338, 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.837 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T01:00:14.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T01:00:14.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.842 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 146.19988584199999, 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T01:00:14.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.844 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:14.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:14.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:14.847 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 95.547597836999998, 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T01:00:14.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T01:00:14.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T01:00:14.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T01:00:14.853 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.853 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.853 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.853 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:138250 3.c 3:32730bfd:::rbd_data.117c1c3044f5.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 53.001900028999998, 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060687680000000001, 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 138250 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758917+0000", 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5550000000000001e-06 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758925+0000", 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5900000000000008e-06 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758947+0000", 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1197000000000001e-05 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758957+0000", 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0137e-05 2024-09-18T01:00:14.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759094+0000", 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013775300000000001 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759246+0000", 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015127900000000001 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:14.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.770839+0000", 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011593538 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819413+0000", 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048573660999999997 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819467+0000", 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3795999999999998e-05 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:14.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819529+0000", 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2675999999999994e-05 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819602+0000", 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2497999999999996e-05 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:14.859 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:14.860 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:00:14.861 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:14.860+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:14.894 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:14.894 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:14.911 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:14.911 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:14.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:14.928 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:14.944 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:14.944 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:14.961 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:14.957+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:14.965 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:14.965 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:14.966 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T01:00:14.966 INFO:teuthology.orchestra.run.smithi042.stderr:marked in osd.3. 2024-09-18T01:00:14.993 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:14.993 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:15.003 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-18T01:00:15.008 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:15.008 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:15.012 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:00:15.030 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:15.050 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:15.051 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:15.062 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:15.061+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:15.147 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:15.147 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:15.161 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:00:15.162 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:15.161+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:15.264 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:15.261+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:15.277 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:15.277 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:15.290 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:00:15.364 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:15.364+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:15.455 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:15.455 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:15.465 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:15.464+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:15.469 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:00:15.567 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:15.565+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:15.593 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:15.593 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:15.608 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:00:15.668 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:15.668+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:15.769 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:15.768+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:15.777 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:15.777 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:15.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:00:15.869 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:15.868+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:15.913 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:15.913 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:15.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:00:15.970 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:15.969+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:16.059 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:16.059 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:16.071 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:16.069+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:16.073 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:00:16.172 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:16.172+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:16.240 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:16.241 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:16.256 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:00:16.272 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:16.272+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:16.373 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:16.372+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:16.382 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:16.382 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:16.395 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:00:16.474 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:16.473+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:16.517 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:16.517 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:16.530 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:00:16.576 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:16.572+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:16.676 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:16.673+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:16.696 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:16.697 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:16.712 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:00:16.777 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:16.773+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:16.844 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:16.844 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:16.858 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:00:16.878 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:16.877+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:16.979 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:16.976+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:16.981 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:16.981 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:16.998 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:00:17.007 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:00:17.070 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:00:17.114 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:17.113+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:17.171 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:17.171 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:17.181 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:17.177+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:17.189 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:00:17.251 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:17.251 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:17.270 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:17.270 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:17.281 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:17.280+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:17.292 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:17.292 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:17.307 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:17.315 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:17.315 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:17.329 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:17.329 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:17.330 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:00:17.351 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:17.351 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:17.369 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:17.369 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:17.387 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:17.420 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:17.420+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:17.483 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:17.480+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:17.496 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:17.496 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:17.496 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:17.496 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:17.496 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:17.507 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:00:17.600 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:17.600+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:17.676 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:17.676 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:17.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:17.676 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:17.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:11010 3.18 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 148.264925409, 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.18636129800000001, 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 11010 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.677 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409211+0000", 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.409215+0000", 2024-09-18T01:00:17.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.737e-06 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472142+0000", 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062927662999999995 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472185+0000", 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472202+0000", 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6742e-05 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472389+0000", 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018678299999999999 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.472837+0000", 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044850000000000001 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.475553+0000", 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027153009999999998 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594709+0000", 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.119156523 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595345+0000", 2024-09-18T01:00:17.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00063591199999999998 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595469+0000", 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012355100000000001 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.595572+0000", 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010363500000000001 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.682 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125622 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 344064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.000790241999994, 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096365904000000002, 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125622 2024-09-18T01:00:17.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673346+0000", 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673347+0000", 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7599999999999997e-07 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673354+0000", 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.486e-06 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673361+0000", 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5520000000000001e-06 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673365+0000", 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7129999999999996e-06 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768923+0000", 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095557673999999995 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.768994+0000", 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0847999999999997e-05 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769712+0000", 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000718055 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125623 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.000765827999999, 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096098244999999999, 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.687 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125623 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673371+0000", 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8200000000000001e-07 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673372+0000", 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5519999999999999e-06 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673376+0000", 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.512e-06 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673379+0000", 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1580000000000001e-06 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769200+0000", 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095820946000000004 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769317+0000", 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117003 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769469+0000", 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015179200000000001 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.691 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125624 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 385024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.000753963999998, 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101719451, 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125624 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673382+0000", 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673383+0000", 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.23e-07 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673384+0000", 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3060000000000001e-06 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673387+0000", 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673390+0000", 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:00:17.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769495+0000", 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.096104599999999998 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.769571+0000", 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6089999999999998e-05 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775102+0000", 2024-09-18T01:00:17.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055311859999999996 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125625 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.000742919999993, 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102242338, 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:17.696 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125625 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673393+0000", 2024-09-18T01:00:17.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673394+0000", 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673395+0000", 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2839999999999999e-06 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673398+0000", 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1379999999999999e-06 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673401+0000", 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7149999999999998e-06 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775140+0000", 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.101739127 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775212+0000", 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1984999999999996e-05 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775636+0000", 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000423752 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125627 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 425984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:17.700 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.000732002999996, 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10248196399999999, 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125627 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:17.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673404+0000", 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673405+0000", 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0699999999999998e-07 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673406+0000", 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2130000000000001e-06 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673409+0000", 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3400000000000002e-06 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673412+0000", 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7360000000000001e-06 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775665+0000", 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.102253552 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775734+0000", 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8019e-05 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.775886+0000", 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015279700000000001 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.704 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125628 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.000720904999994, 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.1031562, 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.705 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125628 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673415+0000", 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673416+0000", 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.91e-07 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673418+0000", 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673423+0000", 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6149999999999996e-06 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673428+0000", 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1460000000000004e-06 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776149+0000", 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10272025899999999 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776218+0000", 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9207000000000003e-05 2024-09-18T01:00:17.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776572+0000", 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035375599999999999 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125629 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 466944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.000701354, 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103390994, 2024-09-18T01:00:17.709 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125629 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673435+0000", 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673438+0000", 2024-09-18T01:00:17.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4310000000000001e-06 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673446+0000", 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0290000000000005e-06 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673451+0000", 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3630000000000002e-06 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776600+0000", 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10314894500000001 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776668+0000", 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7963e-05 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.776826+0000", 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015784499999999999 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.713 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125631 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.000678104000002, 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.104002186, 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.714 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125631 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673458+0000", 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673459+0000", 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673461+0000", 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.351e-06 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673467+0000", 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3389999999999998e-06 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673472+0000", 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9490000000000004e-06 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777062+0000", 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.103589771 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777130+0000", 2024-09-18T01:00:17.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8065999999999995e-05 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777460+0000", 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000330265 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125632 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:17.718 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.000657316000002, 2024-09-18T01:00:17.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10711156300000001, 2024-09-18T01:00:17.719 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.719 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.719 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:17.719 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.719 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.719 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.719 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125632 2024-09-18T01:00:17.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673479+0000", 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6100000000000002e-07 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673482+0000", 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1210000000000001e-06 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673488+0000", 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1659999999999999e-06 2024-09-18T01:00:17.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673493+0000", 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.989e-06 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777489+0000", 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10399652600000001 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.777565+0000", 2024-09-18T01:00:17.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6219999999999999e-05 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780591+0000", 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030251800000000001 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:125633 3.17 3:e9f5a32c:::rbd_data.117c8e0d79e4.000000000000000d:20 [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:17.723 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.000636553999996, 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10735947699999999, 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 125633 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:17.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673500+0000", 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673503+0000", 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4360000000000001e-06 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673509+0000", 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3969999999999999e-06 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.673515+0000", 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7640000000000002e-06 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780622+0000", 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.107107533 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780692+0000", 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9735999999999998e-05 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.780859+0000", 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016721999999999999 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126363 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.962782412999999, 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.10567230599999999, 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126363 2024-09-18T01:00:17.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711354+0000", 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711355+0000", 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9999999999999995e-07 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711360+0000", 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9740000000000001e-06 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711380+0000", 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9769e-05 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.711396+0000", 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5947000000000002e-05 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817011+0000", 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.105615185 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817026+0000", 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5430999999999998e-05 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.731 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126376 3.17 3.6fbaafd7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.952206128, 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.097651948000000002, 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.732 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126376 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721930+0000", 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721931+0000", 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721933+0000", 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2709999999999999e-06 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721939+0000", 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6829999999999996e-06 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721944+0000", 2024-09-18T01:00:17.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0089999999999998e-06 2024-09-18T01:00:17.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817590+0000", 2024-09-18T01:00:17.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095646089000000004 2024-09-18T01:00:17.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.819582+0000", 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019924669999999999 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126427 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.952185595000003, 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095671891999999994, 2024-09-18T01:00:17.736 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126427 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721951+0000", 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9499999999999998e-07 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721954+0000", 2024-09-18T01:00:17.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4700000000000001e-06 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721960+0000", 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4629999999999998e-06 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721965+0000", 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817614+0000", 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095649026999999998 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817623+0000", 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4780000000000008e-06 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126428 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.740 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.952163408000004, 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095683878999999999, 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126428 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721973+0000", 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2899999999999999e-07 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721976+0000", 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4279999999999999e-06 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721983+0000", 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7499999999999997e-06 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721988+0000", 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4349999999999996e-06 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817644+0000", 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095656525000000006 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817657+0000", 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2312e-05 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126429 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.952140368000002, 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095687929000000005, 2024-09-18T01:00:17.744 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126429 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:17.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721996+0000", 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.721999+0000", 2024-09-18T01:00:17.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.649e-06 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722006+0000", 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8299999999999998e-06 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722011+0000", 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6970000000000001e-06 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817678+0000", 2024-09-18T01:00:17.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095667143999999996 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817684+0000", 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2149999999999998e-06 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126430 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.952118244999994, 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095682481999999999, 2024-09-18T01:00:17.748 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126430 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722018+0000", 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722019+0000", 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722021+0000", 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.419e-06 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722027+0000", 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0750000000000003e-06 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722033+0000", 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817695+0000", 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095662607999999996 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817701+0000", 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1359999999999996e-06 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:126431 3.17 3.34c5af97 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.952096342000004, 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095678416000000002, 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.752 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 126431 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722040+0000", 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722043+0000", 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722049+0000", 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3219999999999998e-06 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.722054+0000", 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817715+0000", 2024-09-18T01:00:17.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.095660429000000005 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:08.817718+0000", 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7349999999999998e-06 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160401 3.0 3:0686cf5d:::rbd_data.117cf66ad0cc.0000000000000001:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.602933912000001, 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.89130826799999996, 2024-09-18T01:00:17.756 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160401 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071202+0000", 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071217+0000", 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4924e-05 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071229+0000", 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.166e-05 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071272+0000", 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2506000000000003e-05 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071294+0000", 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2789000000000002e-05 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071664+0000", 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036966100000000002 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:17.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071694+0000", 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0207000000000001e-05 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.955396+0000", 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88370225899999999 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.955864+0000", 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000467997 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.956147+0000", 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028220399999999998 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.960924+0000", 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047771760000000002 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962456+0000", 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001532158 2024-09-18T01:00:17.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962485+0000", 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8758000000000001e-05 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962511+0000", 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5969000000000002e-05 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:160404 3.0 3:048cdd79:::rbd_data.117cf66ad0cc.0000000000000004:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-18T01:00:17.762 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.602756858999999, 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.89253343799999996, 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 160404 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071379+0000", 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071381+0000", 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7829999999999999e-06 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071390+0000", 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.02e-06 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071417+0000", 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6798999999999999e-05 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071434+0000", 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6983e-05 2024-09-18T01:00:17.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071769+0000", 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033536500000000001 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.071788+0000", 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8963000000000001e-05 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961054+0000", 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88926583000000003 2024-09-18T01:00:17.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961171+0000", 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116468 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.961288+0000", 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117759 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.962832+0000", 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001543677 2024-09-18T01:00:17.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963849+0000", 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010164659999999999 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963878+0000", 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9697e-05 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:00.963913+0000", 2024-09-18T01:00:17.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4628000000000001e-05 2024-09-18T01:00:17.769 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.769 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.769 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.769 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:17.769 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:17.769 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:17.770 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:00:17.771 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:17.768+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:17.872 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:17.869+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:17.943 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:17.943 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:17.943 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:17.943 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:17.957 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:00:17.992 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T01:00:18.000 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:18.000+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:18.072 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:18.072+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:18.100 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:18.100 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:18.100 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:18.100 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:18.100 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:18.117 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:00:18.204 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:18.204+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:18.274 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:18.273+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:18.286 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 149.195266917, 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 148.80437196599999, 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T01:00:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T01:00:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141148 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.712760+0000", 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.571120661999998, 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057802683000000001, 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141148 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712760+0000", 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712760+0000", 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712761+0000", 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6729999999999999e-06 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712770+0000", 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6200000000000005e-06 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712791+0000", 2024-09-18T01:00:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.067e-05 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712807+0000", 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5883e-05 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.764648+0000", 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051841127000000001 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.764855+0000", 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020728200000000001 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.765344+0000", 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048933799999999997 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769269+0000", 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0039248900000000003 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770401+0000", 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001132154 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770494+0000", 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2773000000000001e-05 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770562+0000", 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8273000000000002e-05 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141150 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.571040986, 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057991537000000003, 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141150 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712841+0000", 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6330000000000001e-06 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712849+0000", 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1100000000000003e-06 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712873+0000", 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3802999999999999e-05 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712889+0000", 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6184e-05 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766071+0000", 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053181379000000001 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766208+0000", 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013692399999999999 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766800+0000", 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00059252999999999997 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770601+0000", 2024-09-18T01:00:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003800997 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770681+0000", 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9975000000000006e-05 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770770+0000", 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9106999999999999e-05 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770831+0000", 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0894999999999999e-05 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141153 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.57047893, 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060820664000000003, 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141153 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713403+0000", 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6759999999999999e-06 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713411+0000", 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7670000000000003e-06 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713433+0000", 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1807e-05 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713449+0000", 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5795999999999999e-05 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769325+0000", 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055876605000000003 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769426+0000", 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010125500000000001 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769851+0000", 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000424275 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774030+0000", 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041790960000000002 2024-09-18T01:00:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774050+0000", 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.982e-05 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774133+0000", 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3474999999999996e-05 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774222+0000", 2024-09-18T01:00:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9091999999999993e-05 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141154 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.570389507000002, 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062213932999999999, 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141154 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713492+0000", 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.373e-06 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713501+0000", 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6030000000000005e-06 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713538+0000", 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.684e-05 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713555+0000", 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6866000000000001e-05 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771129+0000", 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057574828000000002 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771257+0000", 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012741299999999999 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771699+0000", 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044198799999999998 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775574+0000", 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038754229999999998 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775590+0000", 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6010000000000001e-05 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775655+0000", 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4327999999999998e-05 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775705+0000", 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0260999999999999e-05 2024-09-18T01:00:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141155 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.569824205000003, 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064518352000000001, 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141155 2024-09-18T01:00:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714059+0000", 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.323e-06 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714070+0000", 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.151e-05 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714099+0000", 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9153e-05 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714119+0000", 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9656999999999998e-05 2024-09-18T01:00:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.772354+0000", 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058234701 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.772479+0000", 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012552700000000001 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.773115+0000", 2024-09-18T01:00:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000635831 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775815+0000", 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026998930000000001 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778455+0000", 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026403279999999999 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778510+0000", 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4611999999999998e-05 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778575+0000", 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4817e-05 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141156 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.566780393999998, 2024-09-18T01:00:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061742916000000002, 2024-09-18T01:00:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141156 2024-09-18T01:00:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717103+0000", 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7930000000000002e-06 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717119+0000", 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676999999999999e-05 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717156+0000", 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.701e-05 2024-09-18T01:00:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717178+0000", 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2806000000000002e-05 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774362+0000", 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057183909999999998 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774527+0000", 2024-09-18T01:00:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016523899999999999 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774944+0000", 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041637299999999999 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778643+0000", 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003699174 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778737+0000", 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4007999999999998e-05 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778782+0000", 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4886999999999999e-05 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778843+0000", 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1038999999999996e-05 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141157 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.566663364999997, 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061753200000000001, 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141157 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717219+0000", 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6330000000000001e-06 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717228+0000", 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9539999999999993e-06 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717251+0000", 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3193e-05 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717268+0000", 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.719e-05 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775897+0000", 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058628583999999997 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.776092+0000", 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019526600000000001 2024-09-18T01:00:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.776410+0000", 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031835700000000001 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778719+0000", 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023090910000000001 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778858+0000", 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013843199999999999 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778911+0000", 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2907e-05 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778970+0000", 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9592999999999997e-05 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141158 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.566588547999999, 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064284142000000002, 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141158 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717293+0000", 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3459999999999999e-06 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717322+0000", 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8926000000000001e-05 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717345+0000", 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2779999999999999e-05 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717362+0000", 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7079999999999999e-05 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777123+0000", 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059760675999999999 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777271+0000", 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014837800000000001 2024-09-18T01:00:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777835+0000", 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056387600000000005 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781379+0000", 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035443559999999998 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781444+0000", 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4776000000000005e-05 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781515+0000", 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0533999999999999e-05 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781576+0000", 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1414000000000005e-05 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141159 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.566494982999998, 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065579987000000006, 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141159 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717387+0000", 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717394+0000", 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717413+0000", 2024-09-18T01:00:18.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9004e-05 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717428+0000", 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4780000000000001e-05 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779077+0000", 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061649335999999999 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779225+0000", 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000147374 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779649+0000", 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042398799999999998 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781615+0000", 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001965732 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782884+0000", 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001269879 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782921+0000", 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6927000000000001e-05 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782965+0000", 2024-09-18T01:00:18.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4066999999999997e-05 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141160 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.566431860000002, 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.067301246999999995, 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.351 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141160 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:18.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717450+0000", 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.207e-06 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717456+0000", 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.066e-06 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717468+0000", 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1914e-05 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717482+0000", 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3716e-05 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780389+0000", 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062907819000000004 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780502+0000", 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011259100000000001 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780930+0000", 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042795799999999998 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784605+0000", 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036754719999999999 2024-09-18T01:00:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784631+0000", 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5171999999999999e-05 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784697+0000", 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6506000000000003e-05 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784750+0000", 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2825999999999999e-05 2024-09-18T01:00:18.356 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141165 3.12 3:4e6ddb92:::rbd_data.117c1c3044f5.0000000000000006:27 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.566378637, 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064420416999999994, 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.357 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141165 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717503+0000", 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2529999999999999e-06 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717510+0000", 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717572+0000", 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2476000000000003e-05 2024-09-18T01:00:18.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717588+0000", 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6504e-05 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781669+0000", 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064080210999999998 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781781+0000", 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112137 2024-09-18T01:00:18.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781922+0000", 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014139900000000001 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141172 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.565195819000003, 2024-09-18T01:00:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.067304013999999995, 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141172 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718687+0000", 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5229999999999998e-06 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718701+0000", 2024-09-18T01:00:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3397000000000001e-05 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718728+0000", 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7350000000000001e-05 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718748+0000", 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0205999999999999e-05 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781972+0000", 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.063223835000000006 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782067+0000", 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5206999999999998e-05 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782420+0000", 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035256799999999999 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784838+0000", 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002418023 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785893+0000", 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010552440000000001 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785945+0000", 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1706999999999998e-05 2024-09-18T01:00:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785989+0000", 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3954e-05 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141175 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.563178264000001, 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.066834381999999998, 2024-09-18T01:00:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141175 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720705+0000", 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.79e-06 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720720+0000", 2024-09-18T01:00:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4897e-05 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720754+0000", 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4323000000000003e-05 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720776+0000", 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1535999999999998e-05 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783058+0000", 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062282205 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783203+0000", 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000144816 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783831+0000", 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062834200000000005 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787404+0000", 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035724429999999998 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787419+0000", 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5783000000000001e-05 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787468+0000", 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9107999999999998e-05 2024-09-18T01:00:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787537+0000", 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8139000000000006e-05 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141176 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.563068328, 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.068349919999999995, 2024-09-18T01:00:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141176 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720814+0000", 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.457e-06 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720822+0000", 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5969999999999992e-06 2024-09-18T01:00:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720844+0000", 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1965999999999999e-05 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720860+0000", 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6243e-05 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784925+0000", 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064065009000000006 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785025+0000", 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9854999999999999e-05 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785407+0000", 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038140800000000002 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787604+0000", 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021976389999999999 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789028+0000", 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001423304 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789103+0000", 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5284999999999995e-05 2024-09-18T01:00:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789162+0000", 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9157000000000003e-05 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141177 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.562998475000001, 2024-09-18T01:00:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.069464445999999999, 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141177 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720883+0000", 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.37e-06 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720891+0000", 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8299999999999996e-06 2024-09-18T01:00:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720916+0000", 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5216e-05 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720932+0000", 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676999999999999e-05 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786055+0000", 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065122920000000001 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786179+0000", 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000124153 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786832+0000", 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065315700000000002 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790206+0000", 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003374008 2024-09-18T01:00:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790230+0000", 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3963000000000001e-05 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790291+0000", 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0747e-05 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790346+0000", 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5405000000000002e-05 2024-09-18T01:00:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141179 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.562012885999998, 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.070779396999999994, 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141179 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721870+0000", 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5739999999999998e-06 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721885+0000", 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4443e-05 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721917+0000", 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2122999999999997e-05 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721938+0000", 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1583e-05 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787900+0000", 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065962144 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.788009+0000", 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010851799999999999 2024-09-18T01:00:18.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.788372+0000", 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036301900000000002 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.791319+0000", 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029471609999999998 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792566+0000", 2024-09-18T01:00:18.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001247155 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792615+0000", 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792647+0000", 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1699000000000003e-05 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141180 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.561392114, 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.070290325000000001, 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141180 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722491+0000", 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2024-09-18T01:00:18.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722503+0000", 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2653000000000001e-05 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722532+0000", 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9166000000000002e-05 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722582+0000", 2024-09-18T01:00:18.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789227+0000", 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.066644492999999999 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789286+0000", 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8959999999999998e-05 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789638+0000", 2024-09-18T01:00:18.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035211500000000001 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792662+0000", 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030245369999999999 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792699+0000", 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6810999999999997e-05 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792749+0000", 2024-09-18T01:00:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9885000000000001e-05 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792779+0000", 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9794e-05 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:150977 3.b 3:d4ab7022:::rbd_data.117c741c3c2b.000000000000000c:2a [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.684027385, 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064163676000000003, 2024-09-18T01:00:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 150977 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599854+0000", 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599860+0000", 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5330000000000002e-06 2024-09-18T01:00:18.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599877+0000", 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7056000000000001e-05 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599891+0000", 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4569999999999999e-05 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.652536+0000", 2024-09-18T01:00:18.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052644519000000001 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.652629+0000", 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3255000000000001e-05 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.664017+0000", 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011387516 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.400 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:18.402 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:00:18.402 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:18.400+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:18.503 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:18.500+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:18.575 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:18.575 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:18.575 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:18.576 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:18.589 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:00:18.604 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:18.601+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:18.704 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:18.704+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:18.757 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:18.757 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:18.757 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:18.757 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:18.757 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:18.770 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:00:18.805 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:18.801+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:18.906 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:18.905+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:18.929 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.926262788, 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.930 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.926204531, 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T01:00:18.938 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.92614609200001, 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.939 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T01:00:18.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.92608487499999, 2024-09-18T01:00:18.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T01:00:18.947 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.92599446, 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.948 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T01:00:18.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:18.952 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.92592352700001, 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T01:00:18.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.925863921, 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.957 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T01:00:18.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.925804551, 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T01:00:18.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T01:00:18.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.925742762, 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.966 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:18.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:18.970 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.925609939, 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T01:00:18.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 150.37039246200001, 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.975 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.980 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 150.369892338, 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.981 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 150.36986182999999, 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T01:00:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T01:00:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 150.36983716500001, 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T01:00:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T01:00:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 150.36980863299999, 2024-09-18T01:00:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T01:00:18.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T01:00:18.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:18.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 150.36947534199999, 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T01:00:18.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:19.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 150.369433491, 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:19.003 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:19.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T01:00:19.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T01:00:19.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 150.36839599499999, 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:19.009 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:19.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.013 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 99.716107989999998, 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T01:00:19.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T01:00:19.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:19.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:138250 3.c 3:32730bfd:::rbd_data.117c1c3044f5.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:19.019 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 57.170410181999998, 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060687680000000001, 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 138250 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:19.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758917+0000", 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5550000000000001e-06 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758925+0000", 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5900000000000008e-06 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758947+0000", 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1197000000000001e-05 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758957+0000", 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0137e-05 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759094+0000", 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013775300000000001 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759246+0000", 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015127900000000001 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.770839+0000", 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011593538 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819413+0000", 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048573660999999997 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819467+0000", 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3795999999999998e-05 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819529+0000", 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2675999999999994e-05 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:19.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:19.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:19.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819602+0000", 2024-09-18T01:00:19.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2497999999999996e-05 2024-09-18T01:00:19.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:19.025 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:19.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:19.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:19.025 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:19.025 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:19.027 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:19.024+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:19.027 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:00:19.128 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:19.125+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:19.182 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:19.182 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:19.197 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:00:19.229 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:19.228+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:19.330 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:19.329+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:19.372 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:19.373 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:19.387 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:00:19.407 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:00:19.434 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:19.433+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:19.532 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:19.532+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.572 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.50577268000001, 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T01:00:19.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T01:00:19.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 81.613238860999999, 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.864172070999999, 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.588 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.853797615000005, 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T01:00:19.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.853724932999995, 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T01:00:19.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T01:00:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.853694067000006, 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T01:00:19.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.853641292999995, 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.606 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.853615126999998, 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T01:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T01:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T01:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T01:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.853589971999995, 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T01:00:19.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.853166661000003, 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.616 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T01:00:19.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.853128283000004, 2024-09-18T01:00:19.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:19.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T01:00:19.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.853103517999998, 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T01:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T01:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T01:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T01:00:19.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T01:00:19.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.846308754000006, 2024-09-18T01:00:19.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T01:00:19.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T01:00:19.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.846283154000005, 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.635 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.846259810999996, 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.639 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T01:00:19.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T01:00:19.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.846237372000004, 2024-09-18T01:00:19.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.846216599000002, 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T01:00:19.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T01:00:19.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.846195573000003, 2024-09-18T01:00:19.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T01:00:19.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T01:00:19.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.846154991000006, 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.846120892000002, 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T01:00:19.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T01:00:19.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T01:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T01:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T01:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:19.665 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:19.667 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:00:19.667 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:19.667 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:19.668 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:19.668 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:19.668 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:19.669 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:19.669 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:19.669 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:19.670 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:19.664+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:19.689 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:19.690 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:19.713 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:19.713 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:19.734 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:19.735 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:19.790 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:19.790 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:19.790 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:19.789+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:19.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:00:19.868 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:19.868+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:19.969 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:19.968+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:19.985 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:00:20.004 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6, 7, 3] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:00:20.004 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:00:20.004 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:00:20.004 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:00:20.029 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:00:20.070 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:20.068+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:20.140 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:20.140 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:20.161 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:00:20.171 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:20.168+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:20.271 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:20.269+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:20.285 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:20.285 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:20.299 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:00:20.372 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:20.372+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:20.383 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:00:20.383 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:20.383 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 162, 2024-09-18T01:00:20.383 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "82a96d76-ba83-49e2-a985-84604bd599ba", 2024-09-18T01:00:20.383 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-18T00:57:07.138732+0000", 2024-09-18T01:00:20.383 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-18T01:00:19.986549+0000", 2024-09-18T01:00:20.383 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-18T00:57:20.258605+0000", 2024-09-18T01:00:20.383 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-18T01:00:14.958457+0000", 2024-09-18T01:00:20.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5804032, 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:00:20.384 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "jewel", 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:21.962521+0000", 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-18T01:00:20.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "19", 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:00:20.387 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T01:00:20.389 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.390 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 8, 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 8, 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.25, 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:25.721733+0000", 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:00:20.392 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T01:00:20.393 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "23", 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 23, 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T01:00:20.394 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T01:00:20.395 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T01:00:20.396 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 4.9699997901916504, 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 4.9699997901916504, 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.62999999523162842, 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 3.130000114440918, 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 3.130000114440918, 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T01:00:20.397 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:35.599413+0000", 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:00:20.398 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T01:00:20.399 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "162", 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 45, 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 162, 2024-09-18T01:00:20.400 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:00:20.401 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T01:00:20.402 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 2, 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 2, 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.75, 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.5, 2024-09-18T01:00:20.403 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.5, 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "3f359ce5-6f6e-4161-8554-8ac028347274", 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T01:00:20.404 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 95, 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:20.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-18T01:00:20.407 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:20.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/199763443", 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/199763443", 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/199763443", 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/199763443", 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:20.409 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "48687611-1556-4b6d-86af-7872494a91d7", 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 160, 2024-09-18T01:00:20.410 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.412 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:20.413 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/345149800", 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/345149800", 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/345149800", 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/345149800", 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T01:00:20.415 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "9e6b1f90-ce9b-46d4-92fe-d457f7e415d5", 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-18T01:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-18T01:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-18T01:00:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-18T01:00:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-18T01:00:20.420 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/431239931", 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/431239931", 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/431239931", 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/431239931", 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.421 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5", 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 161, 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.422 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.424 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/1775474180", 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/1775474180", 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/1775474180", 2024-09-18T01:00:20.426 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/1775474180", 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "802ccbeb-7351-4b38-832a-695117c7e493", 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0.8891448974609375, 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:20.427 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 161, 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6808", 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.428 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6809", 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6810", 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.429 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6811", 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6814", 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.430 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6815", 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6812", 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.432 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6813", 2024-09-18T01:00:20.432 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:20.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.432 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.432 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6809/4198401207", 2024-09-18T01:00:20.432 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6811/4198401207", 2024-09-18T01:00:20.432 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6815/4198401207", 2024-09-18T01:00:20.432 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6813/4198401207", 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d958b3da-efa1-4d86-b81d-e78874441437", 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:20.433 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 161, 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6824", 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.434 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6825", 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6826", 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.435 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6827", 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6830", 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.436 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6831", 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6828", 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.437 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6829", 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6825/2599825823", 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6827/2599825823", 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6831/2599825823", 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6829/2599825823", 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "51b3cea8-c686-4fd3-b3f4-ba936cf26b64", 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 161, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:20.439 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6800", 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6801", 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:20.440 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6802", 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6803", 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.441 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6806", 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6807", 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:20.442 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6804", 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6805", 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6801/3550656451", 2024-09-18T01:00:20.443 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6803/3550656451", 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6807/3550656451", 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6805/3550656451", 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b02ead1c-837a-4b07-9ad1-4e903ed5bc4e", 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:20.444 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 161, 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6816", 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:20.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6817", 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6818", 2024-09-18T01:00:20.446 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6819", 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.447 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6822", 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6823", 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:20.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6820", 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6821", 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6817/2993352359", 2024-09-18T01:00:20.449 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6819/2993352359", 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6823/2993352359", 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6821/2993352359", 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:20.450 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.896477+0000", 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.941756+0000", 2024-09-18T01:00:20.451 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.560180+0000", 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T01:00:20.452 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.077394+0000", 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:20.453 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.996264+0000", 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.511686+0000", 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.454 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T01:00:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.411051+0000", 2024-09-18T01:00:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.455 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.888092+0000", 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [ 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.3", 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: 4 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.a", 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: 1 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.457 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.d", 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: 1 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.12", 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: 6 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.13", 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: 1, 2024-09-18T01:00:20.458 INFO:teuthology.orchestra.run.smithi042.stdout: 4 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.16", 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: 5 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.1b", 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: 4 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.1e", 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: 5 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-18T01:00:20.460 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [ 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 22, 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 9 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [ 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 22, 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-18T01:00:20.462 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-18T01:00:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-18T01:00:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-18T01:00:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-18T01:00:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-18T01:00:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:00:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:00:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-18T01:00:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:20.463 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:20.465 INFO:tasks.thrashosds.thrasher:j is {'epoch': 162, 'fsid': '82a96d76-ba83-49e2-a985-84604bd599ba', 'created': '2024-09-18T00:57:07.138732+0000', 'modified': '2024-09-18T01:00:19.986549+0000', 'last_up_change': '2024-09-18T00:57:20.258605+0000', 'last_in_change': '2024-09-18T01:00:14.958457+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T00:57:21.962521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T00:57:25.721733+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.96999979019165, 'score_stable': 4.96999979019165, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 3.130000114440918, 'raw_score_stable': 3.130000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T00:57:35.599413+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 162, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3f359ce5-6f6e-4161-8554-8ac028347274', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 199763443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 199763443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 199763443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 199763443}]}, 'public_addr': '172.21.15.42:6801/199763443', 'cluster_addr': '172.21.15.42:6803/199763443', 'heartbeat_back_addr': '172.21.15.42:6807/199763443', 'heartbeat_front_addr': '172.21.15.42:6805/199763443', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '48687611-1556-4b6d-86af-7872494a91d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 160, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 345149800}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 345149800}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 345149800}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 345149800}]}, 'public_addr': '172.21.15.42:6809/345149800', 'cluster_addr': '172.21.15.42:6811/345149800', 'heartbeat_back_addr': '172.21.15.42:6815/345149800', 'heartbeat_front_addr': '172.21.15.42:6813/345149800', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9e6b1f90-ce9b-46d4-92fe-d457f7e415d5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 431239931}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 431239931}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 431239931}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 431239931}]}, 'public_addr': '172.21.15.42:6825/431239931', 'cluster_addr': '172.21.15.42:6827/431239931', 'heartbeat_back_addr': '172.21.15.42:6831/431239931', 'heartbeat_front_addr': '172.21.15.42:6829/431239931', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 161, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 1775474180}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 1775474180}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 1775474180}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 1775474180}]}, 'public_addr': '172.21.15.42:6817/1775474180', 'cluster_addr': '172.21.15.42:6819/1775474180', 'heartbeat_back_addr': '172.21.15.42:6823/1775474180', 'heartbeat_front_addr': '172.21.15.42:6821/1775474180', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '802ccbeb-7351-4b38-832a-695117c7e493', 'up': 1, 'in': 1, 'weight': 0.8891448974609375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 161, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6809', 'nonce': 4198401207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6811', 'nonce': 4198401207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6815', 'nonce': 4198401207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6813', 'nonce': 4198401207}]}, 'public_addr': '172.21.15.195:6809/4198401207', 'cluster_addr': '172.21.15.195:6811/4198401207', 'heartbeat_back_addr': '172.21.15.195:6815/4198401207', 'heartbeat_front_addr': '172.21.15.195:6813/4198401207', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd958b3da-efa1-4d86-b81d-e78874441437', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 161, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6824', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6825', 'nonce': 2599825823}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6826', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6827', 'nonce': 2599825823}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6830', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6831', 'nonce': 2599825823}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6828', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6829', 'nonce': 2599825823}]}, 'public_addr': '172.21.15.195:6825/2599825823', 'cluster_addr': '172.21.15.195:6827/2599825823', 'heartbeat_back_addr': '172.21.15.195:6831/2599825823', 'heartbeat_front_addr': '172.21.15.195:6829/2599825823', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '51b3cea8-c686-4fd3-b3f4-ba936cf26b64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 161, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6800', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6801', 'nonce': 3550656451}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6803', 'nonce': 3550656451}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6807', 'nonce': 3550656451}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6805', 'nonce': 3550656451}]}, 'public_addr': '172.21.15.195:6801/3550656451', 'cluster_addr': '172.21.15.195:6803/3550656451', 'heartbeat_back_addr': '172.21.15.195:6807/3550656451', 'heartbeat_front_addr': '172.21.15.195:6805/3550656451', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b02ead1c-837a-4b07-9ad1-4e903ed5bc4e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 161, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6817', 'nonce': 2993352359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6818', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6819', 'nonce': 2993352359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6822', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6823', 'nonce': 2993352359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6820', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6821', 'nonce': 2993352359}]}, 'public_addr': '172.21.15.195:6817/2993352359', 'cluster_addr': '172.21.15.195:6819/2993352359', 'heartbeat_back_addr': '172.21.15.195:6823/2993352359', 'heartbeat_front_addr': '172.21.15.195:6821/2993352359', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.896477+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.941756+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.560180+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.077394+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.996264+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.511686+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.411051+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.888092+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.3', 'osds': [4]}, {'pgid': '3.a', 'osds': [1]}, {'pgid': '3.d', 'osds': [1]}, {'pgid': '3.12', 'osds': [6]}, {'pgid': '3.13', 'osds': [1, 4]}, {'pgid': '3.16', 'osds': [5]}, {'pgid': '3.1b', 'osds': [4]}, {'pgid': '3.1e', 'osds': [5]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 22, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 22, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:00:20.465 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-18T01:00:20.465 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:20.465 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:20.473 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:20.472+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:20.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:00:20.574 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:20.573+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:20.635 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:20.635 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:20.649 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:00:20.675 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:20.672+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:20.776 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:20.776+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:20.812 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:20.813 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:20.826 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:00:20.877 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:20.876+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:20.952 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:20.952 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:20.967 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:00:20.995 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T01:00:20.998 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:20.997+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:21.079 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:21.076+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:21.098 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:21.099 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:21.113 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:00:21.180 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:21.177+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:21.245 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:21.245 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:21.259 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:00:21.308 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:21.308+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:21.382 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:21.381+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:21.392 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:21.392 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:21.408 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:00:21.483 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:21.481+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:21.573 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:21.574 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:21.584 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:21.581+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:21.588 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:00:21.685 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:21.685+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:21.711 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:21.712 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:21.728 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:00:21.754 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:00:21.802 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:21.801+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:21.853 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:21.853 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:21.868 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:00:21.916 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:21.916+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:21.927 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:21.927 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:21.951 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:21.951 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:21.973 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:21.973 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:21.988 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:21.985+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:21.991 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:21.991 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:22.004 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:22.005 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:22.015 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:22.015 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:22.019 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:00:22.040 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:22.040 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:22.062 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:22.062 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:22.081 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:22.081 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:22.104 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:22.104+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:22.180 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:22.180 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:22.180 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:22.180 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:22.191 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:22.188+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:22.194 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:00:22.292 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:22.289+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:22.361 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:22.361 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:22.361 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:22.362 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:22.362 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:22.377 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:00:22.392 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:22.389+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:22.494 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:22.493+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:22.502 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:22.502 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:22.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:22.502 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:22.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.41137165000001, 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T01:00:22.506 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 153.020476699, 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T01:00:22.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141150 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.787145719000002, 2024-09-18T01:00:22.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057991537000000003, 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141150 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712841+0000", 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6330000000000001e-06 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712849+0000", 2024-09-18T01:00:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1100000000000003e-06 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712873+0000", 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3802999999999999e-05 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712889+0000", 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6184e-05 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766071+0000", 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053181379000000001 2024-09-18T01:00:22.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766208+0000", 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013692399999999999 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766800+0000", 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00059252999999999997 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770601+0000", 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003800997 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770681+0000", 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9975000000000006e-05 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770770+0000", 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9106999999999999e-05 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770831+0000", 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0894999999999999e-05 2024-09-18T01:00:22.516 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141153 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.786583663000002, 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060820664000000003, 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.517 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141153 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713403+0000", 2024-09-18T01:00:22.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6759999999999999e-06 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713411+0000", 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7670000000000003e-06 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713433+0000", 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1807e-05 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713449+0000", 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5795999999999999e-05 2024-09-18T01:00:22.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769325+0000", 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055876605000000003 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769426+0000", 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010125500000000001 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769851+0000", 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000424275 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774030+0000", 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041790960000000002 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774050+0000", 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.982e-05 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774133+0000", 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3474999999999996e-05 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774222+0000", 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9091999999999993e-05 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.522 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141154 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.786494240000003, 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062213932999999999, 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141154 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713492+0000", 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.373e-06 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713501+0000", 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6030000000000005e-06 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713538+0000", 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.684e-05 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713555+0000", 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6866000000000001e-05 2024-09-18T01:00:22.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771129+0000", 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057574828000000002 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771257+0000", 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012741299999999999 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771699+0000", 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044198799999999998 2024-09-18T01:00:22.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775574+0000", 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038754229999999998 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775590+0000", 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6010000000000001e-05 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775655+0000", 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4327999999999998e-05 2024-09-18T01:00:22.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775705+0000", 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0260999999999999e-05 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141155 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.785928937999998, 2024-09-18T01:00:22.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064518352000000001, 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141155 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714059+0000", 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.323e-06 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714070+0000", 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.151e-05 2024-09-18T01:00:22.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714099+0000", 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9153e-05 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714119+0000", 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9656999999999998e-05 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.772354+0000", 2024-09-18T01:00:22.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058234701 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.772479+0000", 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012552700000000001 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.773115+0000", 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000635831 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775815+0000", 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026998930000000001 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778455+0000", 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026403279999999999 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778510+0000", 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4611999999999998e-05 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778575+0000", 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4817e-05 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141156 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.782885127, 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061742916000000002, 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.534 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141156 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717103+0000", 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7930000000000002e-06 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717119+0000", 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676999999999999e-05 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717156+0000", 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.701e-05 2024-09-18T01:00:22.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717178+0000", 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2806000000000002e-05 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774362+0000", 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057183909999999998 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774527+0000", 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016523899999999999 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774944+0000", 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041637299999999999 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778643+0000", 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003699174 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778737+0000", 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4007999999999998e-05 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778782+0000", 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4886999999999999e-05 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778843+0000", 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1038999999999996e-05 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141157 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.782768097999998, 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061753200000000001, 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141157 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717219+0000", 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6330000000000001e-06 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717228+0000", 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9539999999999993e-06 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717251+0000", 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3193e-05 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717268+0000", 2024-09-18T01:00:22.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.719e-05 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775897+0000", 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058628583999999997 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.776092+0000", 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019526600000000001 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.776410+0000", 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031835700000000001 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778719+0000", 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023090910000000001 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778858+0000", 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013843199999999999 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778911+0000", 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2907e-05 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778970+0000", 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9592999999999997e-05 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.545 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141158 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.782693281, 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064284142000000002, 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141158 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717293+0000", 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3459999999999999e-06 2024-09-18T01:00:22.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717322+0000", 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8926000000000001e-05 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717345+0000", 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2779999999999999e-05 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717362+0000", 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7079999999999999e-05 2024-09-18T01:00:22.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777123+0000", 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059760675999999999 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777271+0000", 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014837800000000001 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777835+0000", 2024-09-18T01:00:22.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056387600000000005 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781379+0000", 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035443559999999998 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781444+0000", 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4776000000000005e-05 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781515+0000", 2024-09-18T01:00:22.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0533999999999999e-05 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781576+0000", 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1414000000000005e-05 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141159 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.782599716, 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065579987000000006, 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.551 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141159 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717387+0000", 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717394+0000", 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717413+0000", 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9004e-05 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717428+0000", 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4780000000000001e-05 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779077+0000", 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061649335999999999 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779225+0000", 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000147374 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779649+0000", 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042398799999999998 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781615+0000", 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001965732 2024-09-18T01:00:22.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782884+0000", 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001269879 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782921+0000", 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6927000000000001e-05 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782965+0000", 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4066999999999997e-05 2024-09-18T01:00:22.556 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141160 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.782536593000003, 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.067301246999999995, 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.557 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141160 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717450+0000", 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.207e-06 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717456+0000", 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.066e-06 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717468+0000", 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1914e-05 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717482+0000", 2024-09-18T01:00:22.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3716e-05 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780389+0000", 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062907819000000004 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780502+0000", 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011259100000000001 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780930+0000", 2024-09-18T01:00:22.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042795799999999998 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784605+0000", 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036754719999999999 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784631+0000", 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5171999999999999e-05 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784697+0000", 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6506000000000003e-05 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784750+0000", 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2825999999999999e-05 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.562 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141165 3.12 3:4e6ddb92:::rbd_data.117c1c3044f5.0000000000000006:27 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.782483370000001, 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064420416999999994, 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141165 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717503+0000", 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2529999999999999e-06 2024-09-18T01:00:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717510+0000", 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717572+0000", 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2476000000000003e-05 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717588+0000", 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6504e-05 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781669+0000", 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064080210999999998 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781781+0000", 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112137 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781922+0000", 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014139900000000001 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.566 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141172 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.781300551999998, 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.067304013999999995, 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141172 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718687+0000", 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5229999999999998e-06 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718701+0000", 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3397000000000001e-05 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718728+0000", 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7350000000000001e-05 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718748+0000", 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0205999999999999e-05 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781972+0000", 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.063223835000000006 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782067+0000", 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5206999999999998e-05 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782420+0000", 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035256799999999999 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784838+0000", 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002418023 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785893+0000", 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010552440000000001 2024-09-18T01:00:22.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785945+0000", 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1706999999999998e-05 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785989+0000", 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3954e-05 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141175 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.779282997000003, 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.066834381999999998, 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141175 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.573 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720705+0000", 2024-09-18T01:00:22.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.79e-06 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720720+0000", 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4897e-05 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720754+0000", 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4323000000000003e-05 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720776+0000", 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1535999999999998e-05 2024-09-18T01:00:22.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783058+0000", 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062282205 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783203+0000", 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000144816 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783831+0000", 2024-09-18T01:00:22.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062834200000000005 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787404+0000", 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035724429999999998 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787419+0000", 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5783000000000001e-05 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787468+0000", 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9107999999999998e-05 2024-09-18T01:00:22.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787537+0000", 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8139000000000006e-05 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141176 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.779173061000002, 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.068349919999999995, 2024-09-18T01:00:22.578 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141176 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720814+0000", 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.457e-06 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720822+0000", 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5969999999999992e-06 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720844+0000", 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1965999999999999e-05 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720860+0000", 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6243e-05 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784925+0000", 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064065009000000006 2024-09-18T01:00:22.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785025+0000", 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9854999999999999e-05 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785407+0000", 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038140800000000002 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787604+0000", 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021976389999999999 2024-09-18T01:00:22.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789028+0000", 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001423304 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789103+0000", 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5284999999999995e-05 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789162+0000", 2024-09-18T01:00:22.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9157000000000003e-05 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141177 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.779103208000002, 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.069464445999999999, 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.584 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141177 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720883+0000", 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.37e-06 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720891+0000", 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8299999999999996e-06 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720916+0000", 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5216e-05 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720932+0000", 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676999999999999e-05 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786055+0000", 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065122920000000001 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786179+0000", 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000124153 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786832+0000", 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065315700000000002 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790206+0000", 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003374008 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790230+0000", 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3963000000000001e-05 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790291+0000", 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0747e-05 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790346+0000", 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5405000000000002e-05 2024-09-18T01:00:22.589 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141179 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.778117619, 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.070779396999999994, 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.590 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141179 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721870+0000", 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5739999999999998e-06 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721885+0000", 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4443e-05 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721917+0000", 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2122999999999997e-05 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721938+0000", 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1583e-05 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787900+0000", 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065962144 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.788009+0000", 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010851799999999999 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.788372+0000", 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036301900000000002 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.791319+0000", 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029471609999999998 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792566+0000", 2024-09-18T01:00:22.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001247155 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792615+0000", 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792647+0000", 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1699000000000003e-05 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.595 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141180 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.777496847000002, 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.070290325000000001, 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141180 2024-09-18T01:00:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722491+0000", 2024-09-18T01:00:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722503+0000", 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2653000000000001e-05 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722532+0000", 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9166000000000002e-05 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722582+0000", 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:00:22.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789227+0000", 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.066644492999999999 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789286+0000", 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8959999999999998e-05 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789638+0000", 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035211500000000001 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792662+0000", 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030245369999999999 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792699+0000", 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6810999999999997e-05 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792749+0000", 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9885000000000001e-05 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792779+0000", 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9794e-05 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:150977 3.b 3:d4ab7022:::rbd_data.117c741c3c2b.000000000000000c:2a [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.900132118000002, 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064163676000000003, 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.601 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 150977 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599854+0000", 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599860+0000", 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5330000000000002e-06 2024-09-18T01:00:22.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599877+0000", 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7056000000000001e-05 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599891+0000", 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4569999999999999e-05 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.652536+0000", 2024-09-18T01:00:22.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052644519000000001 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.652629+0000", 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3255000000000001e-05 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.664017+0000", 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011387516 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:162214 3.1 3:84c67a66:::rbd_header.117cc4c72933:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:18.124688+0000", 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.375297636, 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88222952499999996, 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 162214 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124688+0000", 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124688+0000", 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124691+0000", 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8759999999999998e-06 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124710+0000", 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8813000000000001e-05 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124759+0000", 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9039999999999998e-05 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124789+0000", 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.951e-05 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124889+0000", 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9981000000000004e-05 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124899+0000", 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0416e-05 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:19.006464+0000", 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88156486300000003 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:19.006660+0000", 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000196193 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:19.006917+0000", 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025683299999999999 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:22.610 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:22.612 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:00:22.612 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:22.612+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:22.713 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:22.712+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:22.783 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:22.784 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:22.784 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:22.784 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:22.798 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:00:22.814 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:22.813+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:22.915 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:22.913+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:22.969 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:22.970 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:22.970 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:22.970 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:22.970 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:22.985 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:00:23.015 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:23.013+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:23.019 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:00:23.110 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:23.110 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:23.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:23.110 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:23.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.110 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:23.110 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:23.110 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 158.10483101599999, 2024-09-18T01:00:23.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T01:00:23.110 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.111 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.111 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.111 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T01:00:23.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.111 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T01:00:23.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T01:00:23.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 158.10477275900001, 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.115 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:23.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.119 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 158.10471432, 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T01:00:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 158.104653103, 2024-09-18T01:00:23.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:23.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 158.10456268799999, 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.129 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T01:00:23.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T01:00:23.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 158.104491755, 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.133 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T01:00:23.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T01:00:23.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 158.10443214899999, 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.137 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T01:00:23.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:23.141 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 158.10437277899999, 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.145 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 158.10431098999999, 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T01:00:23.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T01:00:23.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 158.10417816699999, 2024-09-18T01:00:23.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T01:00:23.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 154.54896069, 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.155 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T01:00:23.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T01:00:23.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T01:00:23.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 154.54846056599999, 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.161 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T01:00:23.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 154.54843005800001, 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T01:00:23.165 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T01:00:23.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 154.548405393, 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.170 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:23.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T01:00:23.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.174 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 154.54837686100001, 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T01:00:23.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T01:00:23.178 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 154.54804357, 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.179 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T01:00:23.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 154.54800171900001, 2024-09-18T01:00:23.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T01:00:23.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T01:00:23.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T01:00:23.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 154.546964223, 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:23.189 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:23.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 103.894676218, 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T01:00:23.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T01:00:23.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:138250 3.c 3:32730bfd:::rbd_data.117c1c3044f5.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.348978410000001, 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060687680000000001, 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 138250 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:23.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758917+0000", 2024-09-18T01:00:23.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5550000000000001e-06 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758925+0000", 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5900000000000008e-06 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758947+0000", 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1197000000000001e-05 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758957+0000", 2024-09-18T01:00:23.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0137e-05 2024-09-18T01:00:23.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:23.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759094+0000", 2024-09-18T01:00:23.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013775300000000001 2024-09-18T01:00:23.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:23.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759246+0000", 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015127900000000001 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.770839+0000", 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011593538 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819413+0000", 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048573660999999997 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819467+0000", 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3795999999999998e-05 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819529+0000", 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2675999999999994e-05 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819602+0000", 2024-09-18T01:00:23.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2497999999999996e-05 2024-09-18T01:00:23.206 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.206 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.206 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.206 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:23.206 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:23.206 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:23.207 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:00:23.208 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:23.205+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:23.309 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:23.305+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:23.326 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:23.326 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:23.326 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:23.326 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:23.341 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:00:23.430 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:23.429+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:23.510 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:23.509+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:23.511 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:23.512 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:23.526 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:00:23.614 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:23.613+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:23.703 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:23.703 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:23.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:23.703 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:23.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.703 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:23.703 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:23.703 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.63525564, 2024-09-18T01:00:23.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:23.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 85.742721821000004, 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T01:00:23.709 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.993655031000003, 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.718 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.983280574999995, 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:23.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T01:00:23.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T01:00:23.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.983207892999999, 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.723 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T01:00:23.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T01:00:23.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.983177026999996, 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T01:00:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T01:00:23.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.983124253, 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T01:00:23.731 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T01:00:23.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T01:00:23.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.735 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.983098087000002, 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T01:00:23.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T01:00:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T01:00:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T01:00:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T01:00:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T01:00:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.983072931999999, 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T01:00:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T01:00:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T01:00:23.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T01:00:23.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T01:00:23.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T01:00:23.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.982649620999993, 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T01:00:23.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T01:00:23.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.749 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.982611242999994, 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:23.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.753 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.982586478000002, 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.975791713999996, 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:23.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T01:00:23.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.762 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.975766113999995, 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:23.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T01:00:23.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.975742771, 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T01:00:23.767 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T01:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T01:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T01:00:23.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.975720331999995, 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T01:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T01:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T01:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.975699559000006, 2024-09-18T01:00:23.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:23.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T01:00:23.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T01:00:23.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.975678533000007, 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.782 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T01:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:23.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T01:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T01:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T01:00:23.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T01:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T01:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T01:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T01:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T01:00:23.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.975637950999996, 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T01:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T01:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T01:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T01:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T01:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T01:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T01:00:23.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.975603852000006, 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T01:00:23.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:23.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T01:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T01:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.798 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:23.799 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:00:23.800 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:23.800+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:23.901 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:23.900+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:23.941 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T01:00:23.986 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:23.986 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:23.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.986 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:162461 3.1a 3:59c7b6e4:::rbd_header.117c8de7251e:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e166)", 2024-09-18T01:00:23.986 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T01:00:23.983750+0000", 2024-09-18T01:00:23.986 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0014456440000000001, 2024-09-18T01:00:23.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014672699999999999, 2024-09-18T01:00:23.986 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:23.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:23.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "waiting for sub ops", 2024-09-18T01:00:23.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:23.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:23.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:23.987 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 162461 2024-09-18T01:00:23.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.987 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:23.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:23.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:23.983750+0000", 2024-09-18T01:00:23.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:23.983750+0000", 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:23.983751+0000", 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3900000000000004e-07 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:23.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:23.983756+0000", 2024-09-18T01:00:23.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6639999999999997e-06 2024-09-18T01:00:23.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:23.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:23.983768+0000", 2024-09-18T01:00:23.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.206e-05 2024-09-18T01:00:23.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:23.983774+0000", 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.524e-06 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:23.983813+0000", 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9832000000000001e-05 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:23.983857+0000", 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3553999999999997e-05 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:23.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:23.983953+0000", 2024-09-18T01:00:23.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6107000000000007e-05 2024-09-18T01:00:23.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:23.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:23.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:23.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:23.984605+0000", 2024-09-18T01:00:23.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065248300000000003 2024-09-18T01:00:23.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.992 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:23.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:23.992 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:23.992 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 1 2024-09-18T01:00:23.992 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:24.002 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:24.000+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:24.003 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:00:24.103 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:24.100+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:24.103 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:00:24.178 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:24.178 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:24.179 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:24.179 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:24.194 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:00:24.203 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:24.204+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:24.261 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:24.261 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:24.279 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:24.279 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:24.297 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:24.297 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:24.304 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:24.301+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:24.314 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:24.314 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:24.330 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:24.330 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:24.336 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:24.336 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:24.346 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:00:24.358 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:24.358 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:24.379 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:24.379 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:24.401 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:24.401 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:24.406 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:24.404+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:24.491 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:24.491 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:24.506 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:00:24.507 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:24.504+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:24.608 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:24.605+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:24.631 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:24.631 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:24.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:00:24.709 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:24.708+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:24.810 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:24.805+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:24.813 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:24.813 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:24.827 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:00:24.914 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:24.913+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:24.991 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:24.991 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:25.006 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:00:25.050 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:25.049+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:25.112 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:25.112+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:25.197 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:25.197 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:25.213 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:00:25.214 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:25.209+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:25.314 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:25.312+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:25.402 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:00:25.448 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:25.448+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:25.466 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6, 7, 3] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:00:25.466 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:00:25.466 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-18T01:00:25.466 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-18T01:00:25.515 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:25.513+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:25.534 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:25.640 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:25.641 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:00:25.642 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:25.640+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:25.742 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:25.741+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:25.781 INFO:teuthology.orchestra.run.smithi042.stdout:moved 0 / 82 (0%) 2024-09-18T01:00:25.781 INFO:teuthology.orchestra.run.smithi042.stdout:avg 13.6667 2024-09-18T01:00:25.781 INFO:teuthology.orchestra.run.smithi042.stdout:stddev 4.6428 -> 4.6428 (expected baseline 3.37474) 2024-09-18T01:00:25.781 INFO:teuthology.orchestra.run.smithi042.stdout:min osd.7 with 7 -> 7 pgs (0.512195 -> 0.512195 * mean) 2024-09-18T01:00:25.781 INFO:teuthology.orchestra.run.smithi042.stdout:max osd.3 with 22 -> 22 pgs (1.60976 -> 1.60976 * mean) 2024-09-18T01:00:25.782 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:00:25.782 INFO:teuthology.orchestra.run.smithi042.stdout:oload 1000 2024-09-18T01:00:25.782 INFO:teuthology.orchestra.run.smithi042.stdout:max_change 0.05 2024-09-18T01:00:25.782 INFO:teuthology.orchestra.run.smithi042.stdout:max_change_osds 4 2024-09-18T01:00:25.782 INFO:teuthology.orchestra.run.smithi042.stdout:average_utilization 0.0012 2024-09-18T01:00:25.782 INFO:teuthology.orchestra.run.smithi042.stdout:overload_utilization 0.0118 2024-09-18T01:00:25.782 INFO:teuthology.orchestra.run.smithi042.stderr:no change 2024-09-18T01:00:25.800 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T01:00:25.800 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-18T01:00:25.811 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:25.811 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:25.825 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:00:25.872 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:25.872+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:25.944 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:25.944 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:25.945 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:25.941+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:25.958 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:00:25.970 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:00:26.046 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:26.045+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:26.141 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:26.141 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:26.146 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:26.144+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:26.157 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:00:26.179 INFO:teuthology.orchestra.run.smithi042.stdout:moved 0 / 82 (0%) 2024-09-18T01:00:26.179 INFO:teuthology.orchestra.run.smithi042.stdout:avg 13.6667 2024-09-18T01:00:26.179 INFO:teuthology.orchestra.run.smithi042.stdout:stddev 4.6428 -> 4.6428 (expected baseline 3.37474) 2024-09-18T01:00:26.179 INFO:teuthology.orchestra.run.smithi042.stdout:min osd.7 with 7 -> 7 pgs (0.512195 -> 0.512195 * mean) 2024-09-18T01:00:26.179 INFO:teuthology.orchestra.run.smithi042.stdout:max osd.3 with 22 -> 22 pgs (1.60976 -> 1.60976 * mean) 2024-09-18T01:00:26.179 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:00:26.179 INFO:teuthology.orchestra.run.smithi042.stdout:oload 1000 2024-09-18T01:00:26.179 INFO:teuthology.orchestra.run.smithi042.stdout:max_change 1 2024-09-18T01:00:26.179 INFO:teuthology.orchestra.run.smithi042.stdout:max_change_osds 4 2024-09-18T01:00:26.179 INFO:teuthology.orchestra.run.smithi042.stdout:average_utilization 0.0010 2024-09-18T01:00:26.179 INFO:teuthology.orchestra.run.smithi042.stdout:overload_utilization 0.0100 2024-09-18T01:00:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:no change 2024-09-18T01:00:26.200 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T01:00:26.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-18T01:00:26.247 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:26.244+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:26.332 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:26.332 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:26.348 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:00:26.348 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:26.348+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:26.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:00:26.466 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:26.465+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:26.514 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:26.515 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:26.529 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:00:26.559 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:26.559 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:26.575 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:26.575 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:26.576 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:26.576+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:26.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:26.591 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:26.606 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:26.607 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:26.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:26.625 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:26.643 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:26.643 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:26.650 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:26.648+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:26.653 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:26.653 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:26.663 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:26.668 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:00:26.682 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:26.683 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:26.750 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:26.749+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:26.833 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:26.833 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:26.833 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:26.833 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:26.833 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:26.847 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:00:26.896 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:26.896+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:26.928 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T01:00:26.928 INFO:teuthology.orchestra.run.smithi042.stdout:moved 5 / 82 (6.09756%) 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout:avg 13.6667 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout:stddev 4.6428 -> 2.86744 (expected baseline 3.37474) 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout:min osd.7 with 7 -> 9 pgs (0.512195 -> 0.658537 * mean) 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout:max osd.3 with 22 -> 18 pgs (1.60976 -> 1.31707 * mean) 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout:oload 110 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout:max_change 3 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout:max_change_osds 4 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout:average_utilization 0.0010 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout:overload_utilization 0.0011 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 weight 1.0000 -> 0.8141 2024-09-18T01:00:26.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 weight 0.8891 -> 0.7792 2024-09-18T01:00:26.952 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:26.949+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:26.957 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T01:00:26.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-18T01:00:26.981 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:26.982 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:26.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:26.982 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:26.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.982 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:9807 3.15 e34/24 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head v 34'39, mlcod=34'37)", 2024-09-18T01:00:26.982 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:26.983 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 157.89165715199999, 2024-09-18T01:00:26.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114752518, 2024-09-18T01:00:26.983 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:26.983 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:26.983 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:26.983 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:26.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:26.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:26.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:26.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:26.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088614+0000", 2024-09-18T01:00:26.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:26.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:26.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.088616+0000", 2024-09-18T01:00:26.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.446e-06 2024-09-18T01:00:26.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:26.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128009+0000", 2024-09-18T01:00:26.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039392714000000002 2024-09-18T01:00:26.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:26.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128052+0000", 2024-09-18T01:00:26.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2024-09-18T01:00:26.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:26.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128070+0000", 2024-09-18T01:00:26.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7900000000000001e-05 2024-09-18T01:00:26.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:26.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128206+0000", 2024-09-18T01:00:26.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136091 2024-09-18T01:00:26.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.128248+0000", 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1423e-05 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203218+0000", 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074970464000000001 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.203366+0000", 2024-09-18T01:00:26.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014768600000000001 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4476.0:11010 3.18 e34/24 3:1f554377:::rbd_data.117c918d6242.0000000000000001:head v 34'54, mlcod=33'52)", 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 157.50076220099999, 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.114858258, 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:26.992 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479508+0000", 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.479511+0000", 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.384e-06 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511927+0000", 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032416424999999999 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511974+0000", 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7049000000000003e-05 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.511993+0000", 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8867000000000001e-05 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512140+0000", 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146649 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.512201+0000", 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1276000000000001e-05 2024-09-18T01:00:26.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594249+0000", 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.082047731999999998 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:49.594367+0000", 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011787599999999999 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141150 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.267431221000002, 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057991537000000003, 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141150 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:26.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712839+0000", 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712841+0000", 2024-09-18T01:00:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6330000000000001e-06 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712849+0000", 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1100000000000003e-06 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712873+0000", 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3802999999999999e-05 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:26.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.712889+0000", 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6184e-05 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766071+0000", 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053181379000000001 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766208+0000", 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013692399999999999 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.766800+0000", 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00059252999999999997 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770601+0000", 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003800997 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770681+0000", 2024-09-18T01:00:27.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9975000000000006e-05 2024-09-18T01:00:27.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770770+0000", 2024-09-18T01:00:27.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9106999999999999e-05 2024-09-18T01:00:27.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.770831+0000", 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0894999999999999e-05 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141153 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.266869165000003, 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060820664000000003, 2024-09-18T01:00:27.003 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141153 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713402+0000", 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713403+0000", 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6759999999999999e-06 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713411+0000", 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7670000000000003e-06 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713433+0000", 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1807e-05 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713449+0000", 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5795999999999999e-05 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769325+0000", 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055876605000000003 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769426+0000", 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010125500000000001 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.769851+0000", 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000424275 2024-09-18T01:00:27.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774030+0000", 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041790960000000002 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774050+0000", 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.982e-05 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774133+0000", 2024-09-18T01:00:27.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3474999999999996e-05 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774222+0000", 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9091999999999993e-05 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141154 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.009 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.266779741999997, 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062213932999999999, 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141154 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713491+0000", 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713492+0000", 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.373e-06 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713501+0000", 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6030000000000005e-06 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713538+0000", 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.684e-05 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.713555+0000", 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6866000000000001e-05 2024-09-18T01:00:27.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771129+0000", 2024-09-18T01:00:27.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057574828000000002 2024-09-18T01:00:27.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771257+0000", 2024-09-18T01:00:27.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012741299999999999 2024-09-18T01:00:27.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.771699+0000", 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044198799999999998 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775574+0000", 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038754229999999998 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775590+0000", 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6010000000000001e-05 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775655+0000", 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4327999999999998e-05 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775705+0000", 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0260999999999999e-05 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.016 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141155 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.016 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:27.016 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.266214439999999, 2024-09-18T01:00:27.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064518352000000001, 2024-09-18T01:00:27.016 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.016 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.016 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.016 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141155 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714056+0000", 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714059+0000", 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.323e-06 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714070+0000", 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.151e-05 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714099+0000", 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9153e-05 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.714119+0000", 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9656999999999998e-05 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.772354+0000", 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058234701 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.772479+0000", 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012552700000000001 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.773115+0000", 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000635831 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775815+0000", 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026998930000000001 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778455+0000", 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026403279999999999 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778510+0000", 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4611999999999998e-05 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778575+0000", 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4817e-05 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.021 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141156 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.263170629000001, 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061742916000000002, 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141156 2024-09-18T01:00:27.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717100+0000", 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717103+0000", 2024-09-18T01:00:27.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7930000000000002e-06 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717119+0000", 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676999999999999e-05 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717156+0000", 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.701e-05 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717178+0000", 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2806000000000002e-05 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774362+0000", 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057183909999999998 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774527+0000", 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016523899999999999 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.774944+0000", 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041637299999999999 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778643+0000", 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003699174 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778737+0000", 2024-09-18T01:00:27.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4007999999999998e-05 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778782+0000", 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4886999999999999e-05 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778843+0000", 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1038999999999996e-05 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.027 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141157 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.263053599999999, 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061753200000000001, 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.028 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141157 2024-09-18T01:00:27.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.029 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:27.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717217+0000", 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717219+0000", 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6330000000000001e-06 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717228+0000", 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9539999999999993e-06 2024-09-18T01:00:27.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717251+0000", 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3193e-05 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717268+0000", 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.719e-05 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.775897+0000", 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.058628583999999997 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.776092+0000", 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019526600000000001 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.776410+0000", 2024-09-18T01:00:27.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031835700000000001 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778719+0000", 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023090910000000001 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778858+0000", 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013843199999999999 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778911+0000", 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2907e-05 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.778970+0000", 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9592999999999997e-05 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141158 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.262978783000001, 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064284142000000002, 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141158 2024-09-18T01:00:27.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717292+0000", 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717293+0000", 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3459999999999999e-06 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717322+0000", 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8926000000000001e-05 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717345+0000", 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2779999999999999e-05 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717362+0000", 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7079999999999999e-05 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777123+0000", 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.059760675999999999 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777271+0000", 2024-09-18T01:00:27.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014837800000000001 2024-09-18T01:00:27.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.777835+0000", 2024-09-18T01:00:27.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056387600000000005 2024-09-18T01:00:27.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781379+0000", 2024-09-18T01:00:27.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035443559999999998 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781444+0000", 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4776000000000005e-05 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781515+0000", 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0533999999999999e-05 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781576+0000", 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1414000000000005e-05 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141159 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.262885218000001, 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065579987000000006, 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.041 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141159 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717385+0000", 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717387+0000", 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717394+0000", 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6529999999999994e-06 2024-09-18T01:00:27.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717413+0000", 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9004e-05 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717428+0000", 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4780000000000001e-05 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779077+0000", 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.061649335999999999 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779225+0000", 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000147374 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.779649+0000", 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042398799999999998 2024-09-18T01:00:27.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781615+0000", 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001965732 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782884+0000", 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001269879 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782921+0000", 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6927000000000001e-05 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782965+0000", 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4066999999999997e-05 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.047 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141160 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.262822094999997, 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.067301246999999995, 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141160 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717449+0000", 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717450+0000", 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.207e-06 2024-09-18T01:00:27.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717456+0000", 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.066e-06 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717468+0000", 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1914e-05 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717482+0000", 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3716e-05 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780389+0000", 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062907819000000004 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780502+0000", 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011259100000000001 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.780930+0000", 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042795799999999998 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784605+0000", 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036754719999999999 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784631+0000", 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5171999999999999e-05 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784697+0000", 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6506000000000003e-05 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784750+0000", 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2825999999999999e-05 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.053 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141165 3.12 3:4e6ddb92:::rbd_data.117c1c3044f5.0000000000000006:27 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.262768872000002, 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064420416999999994, 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141165 2024-09-18T01:00:27.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717502+0000", 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717503+0000", 2024-09-18T01:00:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2529999999999999e-06 2024-09-18T01:00:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717510+0000", 2024-09-18T01:00:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4370000000000004e-06 2024-09-18T01:00:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717572+0000", 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2476000000000003e-05 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.717588+0000", 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6504e-05 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781669+0000", 2024-09-18T01:00:27.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064080210999999998 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781781+0000", 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000112137 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781922+0000", 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014139900000000001 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141172 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.261586053999999, 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.067304013999999995, 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141172 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.059 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718685+0000", 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718687+0000", 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5229999999999998e-06 2024-09-18T01:00:27.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718701+0000", 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3397000000000001e-05 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718728+0000", 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7350000000000001e-05 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.718748+0000", 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0205999999999999e-05 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.781972+0000", 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.063223835000000006 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782067+0000", 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5206999999999998e-05 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.782420+0000", 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035256799999999999 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784838+0000", 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002418023 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785893+0000", 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010552440000000001 2024-09-18T01:00:27.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785945+0000", 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1706999999999998e-05 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785989+0000", 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3954e-05 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141175 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.259568498999997, 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.066834381999999998, 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141175 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.065 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720702+0000", 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720705+0000", 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.79e-06 2024-09-18T01:00:27.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720720+0000", 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4897e-05 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720754+0000", 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4323000000000003e-05 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720776+0000", 2024-09-18T01:00:27.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1535999999999998e-05 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783058+0000", 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.062282205 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783203+0000", 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000144816 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.783831+0000", 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062834200000000005 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787404+0000", 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035724429999999998 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787419+0000", 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5783000000000001e-05 2024-09-18T01:00:27.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787468+0000", 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9107999999999998e-05 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787537+0000", 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8139000000000006e-05 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141176 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.259458563000003, 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.068349919999999995, 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141176 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720812+0000", 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720814+0000", 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.457e-06 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720822+0000", 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5969999999999992e-06 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720844+0000", 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1965999999999999e-05 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720860+0000", 2024-09-18T01:00:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6243e-05 2024-09-18T01:00:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.784925+0000", 2024-09-18T01:00:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064065009000000006 2024-09-18T01:00:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785025+0000", 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9854999999999999e-05 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.785407+0000", 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038140800000000002 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787604+0000", 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021976389999999999 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789028+0000", 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001423304 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789103+0000", 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5284999999999995e-05 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789162+0000", 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9157000000000003e-05 2024-09-18T01:00:27.076 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141177 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.259388710000003, 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.069464445999999999, 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.077 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141177 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720882+0000", 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720883+0000", 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.37e-06 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720891+0000", 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8299999999999996e-06 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720916+0000", 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5216e-05 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.720932+0000", 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676999999999999e-05 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786055+0000", 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065122920000000001 2024-09-18T01:00:27.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786179+0000", 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000124153 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.786832+0000", 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065315700000000002 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790206+0000", 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003374008 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790230+0000", 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3963000000000001e-05 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790291+0000", 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0747e-05 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.790346+0000", 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5405000000000002e-05 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141179 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.258403121000001, 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.070779396999999994, 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.083 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141179 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721868+0000", 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721870+0000", 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5739999999999998e-06 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721885+0000", 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4443e-05 2024-09-18T01:00:27.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721917+0000", 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2122999999999997e-05 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.721938+0000", 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1583e-05 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.787900+0000", 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.065962144 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.788009+0000", 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010851799999999999 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.788372+0000", 2024-09-18T01:00:27.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036301900000000002 2024-09-18T01:00:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.791319+0000", 2024-09-18T01:00:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029471609999999998 2024-09-18T01:00:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792566+0000", 2024-09-18T01:00:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001247155 2024-09-18T01:00:27.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792615+0000", 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8977999999999998e-05 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792647+0000", 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1699000000000003e-05 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:141180 3.12 3:4e7fffc7:::rbd_object_map.117c8a4c380f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.257782349000003, 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.070290325000000001, 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 141180 2024-09-18T01:00:27.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722488+0000", 2024-09-18T01:00:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722491+0000", 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722503+0000", 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2653000000000001e-05 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722532+0000", 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9166000000000002e-05 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.722582+0000", 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9626999999999998e-05 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789227+0000", 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.066644492999999999 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789286+0000", 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8959999999999998e-05 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.789638+0000", 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035211500000000001 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792662+0000", 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030245369999999999 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792699+0000", 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6810999999999997e-05 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792749+0000", 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9885000000000001e-05 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:32.792779+0000", 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9794e-05 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:150977 3.b 3:d4ab7022:::rbd_data.117c741c3c2b.000000000000000c:2a [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124)", 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.380417620000003, 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.064163676000000003, 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.096 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 150977 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599853+0000", 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599854+0000", 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2270000000000001e-06 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599860+0000", 2024-09-18T01:00:27.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5330000000000002e-06 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599877+0000", 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7056000000000001e-05 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.599891+0000", 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4569999999999999e-05 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.652536+0000", 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052644519000000001 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.652629+0000", 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3255000000000001e-05 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:40.664017+0000", 2024-09-18T01:00:27.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011387516 2024-09-18T01:00:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:162214 3.1 3:84c67a66:::rbd_header.117cc4c72933:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T01:00:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T01:00:18.124688+0000", 2024-09-18T01:00:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.8555831380000001, 2024-09-18T01:00:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88222952499999996, 2024-09-18T01:00:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.101 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 162214 2024-09-18T01:00:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124688+0000", 2024-09-18T01:00:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124688+0000", 2024-09-18T01:00:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124691+0000", 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8759999999999998e-06 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124710+0000", 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8813000000000001e-05 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124759+0000", 2024-09-18T01:00:27.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9039999999999998e-05 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124789+0000", 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.951e-05 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124889+0000", 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9981000000000004e-05 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-18T01:00:27.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:18.124899+0000", 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0416e-05 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:19.006464+0000", 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.88156486300000003 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:19.006660+0000", 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000196193 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T01:00:19.006917+0000", 2024-09-18T01:00:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025683299999999999 2024-09-18T01:00:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.107 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.107 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:27.109 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:00:27.109 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:27.108+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:27.210 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:27.209+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:27.283 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:27.284 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:27.284 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:27.284 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:27.297 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:00:27.344 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:27.344+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:27.400 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T01:00:27.400+0000 7f72bb180640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-18T01:00:27.401 INFO:teuthology.orchestra.run.smithi042.stdout:moved 4 / 82 (4.87805%) 2024-09-18T01:00:27.401 INFO:teuthology.orchestra.run.smithi042.stdout:avg 13.6667 2024-09-18T01:00:27.401 INFO:teuthology.orchestra.run.smithi042.stdout:stddev 2.86744 -> 1.49071 (expected baseline 3.37474) 2024-09-18T01:00:27.401 INFO:teuthology.orchestra.run.smithi042.stdout:min osd.7 with 9 -> 12 pgs (0.658537 -> 0.878049 * mean) 2024-09-18T01:00:27.402 INFO:teuthology.orchestra.run.smithi042.stdout:max osd.3 with 18 -> 15 pgs (1.31707 -> 1.09756 * mean) 2024-09-18T01:00:27.402 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:00:27.402 INFO:teuthology.orchestra.run.smithi042.stdout:oload 110 2024-09-18T01:00:27.402 INFO:teuthology.orchestra.run.smithi042.stdout:max_change 3 2024-09-18T01:00:27.402 INFO:teuthology.orchestra.run.smithi042.stdout:max_change_osds 4 2024-09-18T01:00:27.402 INFO:teuthology.orchestra.run.smithi042.stdout:average_utilization 0.0010 2024-09-18T01:00:27.402 INFO:teuthology.orchestra.run.smithi042.stdout:overload_utilization 0.0011 2024-09-18T01:00:27.402 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 weight 0.8141 -> 0.6627 2024-09-18T01:00:27.402 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 weight 0.7792 -> 0.6828 2024-09-18T01:00:27.403 INFO:teuthology.orchestra.run.smithi042.stderr:no change 2024-09-18T01:00:27.411 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:27.408+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:27.415 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:27.415 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:27.415 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:27.415 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:27.415 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:27.418 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T01:00:27.418 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-18T01:00:27.430 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:00:27.516 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:27.516+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:27.590 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:27.590 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:27.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:27.590 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:27.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.591 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:27.591 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:27.591 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 162.58650329599999, 2024-09-18T01:00:27.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T01:00:27.591 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.591 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.591 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.592 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T01:00:27.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.592 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:27.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:27.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T01:00:27.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T01:00:27.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T01:00:27.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T01:00:27.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T01:00:27.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T01:00:27.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T01:00:27.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T01:00:27.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T01:00:27.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T01:00:27.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T01:00:27.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T01:00:27.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T01:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T01:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:27.600 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 162.58644503900001, 2024-09-18T01:00:27.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T01:00:27.600 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.600 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.600 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.600 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.601 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.601 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.601 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T01:00:27.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.601 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:27.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:27.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T01:00:27.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T01:00:27.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T01:00:27.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T01:00:27.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T01:00:27.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.605 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.605 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 162.5863866, 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:27.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:27.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T01:00:27.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T01:00:27.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T01:00:27.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T01:00:27.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T01:00:27.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T01:00:27.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T01:00:27.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T01:00:27.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T01:00:27.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T01:00:27.611 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.611 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.611 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.611 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:27.611 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:27.611 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 162.586325383, 2024-09-18T01:00:27.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:27.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T01:00:27.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T01:00:27.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T01:00:27.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T01:00:27.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T01:00:27.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T01:00:27.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T01:00:27.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T01:00:27.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.616 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:27.617 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:27.617 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 162.58623496800001, 2024-09-18T01:00:27.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T01:00:27.617 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.617 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.617 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.617 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.617 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.617 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.617 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T01:00:27.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T01:00:27.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T01:00:27.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T01:00:27.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T01:00:27.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T01:00:27.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T01:00:27.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T01:00:27.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T01:00:27.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T01:00:27.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T01:00:27.621 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.622 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.622 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.622 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:27.622 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:27.622 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 162.586164035, 2024-09-18T01:00:27.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T01:00:27.622 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.622 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.622 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:27.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T01:00:27.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T01:00:27.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T01:00:27.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 162.58610442899999, 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T01:00:27.627 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:27.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:27.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T01:00:27.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T01:00:27.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T01:00:27.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T01:00:27.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T01:00:27.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T01:00:27.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T01:00:27.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T01:00:27.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T01:00:27.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T01:00:27.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T01:00:27.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T01:00:27.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T01:00:27.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T01:00:27.632 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.632 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.632 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.633 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:27.633 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:27.633 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 162.58604505900001, 2024-09-18T01:00:27.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T01:00:27.633 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.633 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.633 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.633 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.633 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T01:00:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:27.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T01:00:27.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T01:00:27.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T01:00:27.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T01:00:27.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T01:00:27.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T01:00:27.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T01:00:27.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T01:00:27.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T01:00:27.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T01:00:27.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T01:00:27.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T01:00:27.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T01:00:27.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T01:00:27.638 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.638 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.638 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.638 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:27.638 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 162.58598327000001, 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:27.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:27.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T01:00:27.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T01:00:27.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T01:00:27.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T01:00:27.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T01:00:27.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T01:00:27.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T01:00:27.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T01:00:27.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T01:00:27.643 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.643 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.644 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.644 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:27.644 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:27.644 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 162.58585044700001, 2024-09-18T01:00:27.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T01:00:27.644 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.644 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.644 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.644 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:27.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T01:00:27.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T01:00:27.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T01:00:27.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T01:00:27.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T01:00:27.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T01:00:27.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T01:00:27.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T01:00:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T01:00:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 159.03063297, 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:27.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:27.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T01:00:27.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T01:00:27.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T01:00:27.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T01:00:27.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T01:00:27.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T01:00:27.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T01:00:27.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T01:00:27.656 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.656 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.656 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.656 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:27.656 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:27.657 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 159.03013284599999, 2024-09-18T01:00:27.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T01:00:27.657 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.657 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.657 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.657 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.657 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.657 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.657 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T01:00:27.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.657 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T01:00:27.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T01:00:27.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T01:00:27.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T01:00:27.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T01:00:27.661 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.661 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.662 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:27.662 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:27.662 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 159.03010233800001, 2024-09-18T01:00:27.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T01:00:27.662 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.662 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.662 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.662 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:27.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:27.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T01:00:27.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T01:00:27.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T01:00:27.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T01:00:27.667 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.667 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.667 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.667 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:27.667 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:27.667 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 159.03007767299999, 2024-09-18T01:00:27.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T01:00:27.668 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.668 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.668 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.668 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.668 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.668 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.668 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T01:00:27.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.668 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:27.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:27.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T01:00:27.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T01:00:27.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T01:00:27.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T01:00:27.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T01:00:27.672 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.672 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.672 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 159.03004914100001, 2024-09-18T01:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T01:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T01:00:27.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T01:00:27.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T01:00:27.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T01:00:27.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T01:00:27.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T01:00:27.678 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.678 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.678 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.678 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:27.678 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:27.678 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 159.02971585, 2024-09-18T01:00:27.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:27.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:27.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T01:00:27.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T01:00:27.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T01:00:27.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T01:00:27.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T01:00:27.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T01:00:27.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 159.02967399900001, 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T01:00:27.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.685 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:27.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:27.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T01:00:27.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T01:00:27.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T01:00:27.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T01:00:27.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:27.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T01:00:27.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T01:00:27.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T01:00:27.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T01:00:27.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T01:00:27.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T01:00:27.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.691 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:27.691 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:27.691 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 159.028636503, 2024-09-18T01:00:27.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T01:00:27.691 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.691 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.692 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:27.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.692 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.692 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T01:00:27.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.692 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:27.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T01:00:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T01:00:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T01:00:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T01:00:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T01:00:27.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T01:00:27.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T01:00:27.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T01:00:27.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.696 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T01:00:27.697 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:27.697 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 108.376348498, 2024-09-18T01:00:27.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T01:00:27.697 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.697 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.697 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.697 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.697 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.697 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.697 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T01:00:27.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.698 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:27.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:27.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T01:00:27.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T01:00:27.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T01:00:27.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T01:00:27.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T01:00:27.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T01:00:27.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T01:00:27.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T01:00:27.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:27.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T01:00:27.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T01:00:27.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T01:00:27.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T01:00:27.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T01:00:27.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.704 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.704 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.704 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:138250 3.c 3:32730bfd:::rbd_data.117c1c3044f5.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-18T01:00:27.704 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:27.704 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.830650689999999, 2024-09-18T01:00:27.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060687680000000001, 2024-09-18T01:00:27.704 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:27.704 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:27.704 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:27.705 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:27.705 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:27.705 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:27.705 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 138250 2024-09-18T01:00:27.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.705 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:27.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:27.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:27.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:27.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:27.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:27.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:27.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758917+0000", 2024-09-18T01:00:27.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5550000000000001e-06 2024-09-18T01:00:27.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:27.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758925+0000", 2024-09-18T01:00:27.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5900000000000008e-06 2024-09-18T01:00:27.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:27.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758947+0000", 2024-09-18T01:00:27.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1197000000000001e-05 2024-09-18T01:00:27.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:27.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758957+0000", 2024-09-18T01:00:27.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0137e-05 2024-09-18T01:00:27.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:27.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759094+0000", 2024-09-18T01:00:27.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013775300000000001 2024-09-18T01:00:27.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:27.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759246+0000", 2024-09-18T01:00:27.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015127900000000001 2024-09-18T01:00:27.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:27.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.770839+0000", 2024-09-18T01:00:27.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011593538 2024-09-18T01:00:27.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:27.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819413+0000", 2024-09-18T01:00:27.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048573660999999997 2024-09-18T01:00:27.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:27.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819467+0000", 2024-09-18T01:00:27.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3795999999999998e-05 2024-09-18T01:00:27.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:27.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819529+0000", 2024-09-18T01:00:27.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2675999999999994e-05 2024-09-18T01:00:27.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:27.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:27.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:27.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819602+0000", 2024-09-18T01:00:27.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2497999999999996e-05 2024-09-18T01:00:27.712 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.712 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.712 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.712 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:27.712 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:27.712 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:27.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:00:27.715 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:27.713+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:27.761 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-18T01:00:27.760+0000 7f72bb180640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-18T01:00:27.762 INFO:teuthology.orchestra.run.smithi042.stdout:moved 4 / 82 (4.87805%) 2024-09-18T01:00:27.762 INFO:teuthology.orchestra.run.smithi042.stdout:avg 13.6667 2024-09-18T01:00:27.762 INFO:teuthology.orchestra.run.smithi042.stdout:stddev 2.86744 -> 1.49071 (expected baseline 3.37474) 2024-09-18T01:00:27.762 INFO:teuthology.orchestra.run.smithi042.stdout:min osd.7 with 9 -> 12 pgs (0.658537 -> 0.878049 * mean) 2024-09-18T01:00:27.762 INFO:teuthology.orchestra.run.smithi042.stdout:max osd.3 with 18 -> 15 pgs (1.31707 -> 1.09756 * mean) 2024-09-18T01:00:27.762 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:00:27.762 INFO:teuthology.orchestra.run.smithi042.stdout:oload 110 2024-09-18T01:00:27.762 INFO:teuthology.orchestra.run.smithi042.stdout:max_change 3 2024-09-18T01:00:27.763 INFO:teuthology.orchestra.run.smithi042.stdout:max_change_osds 4 2024-09-18T01:00:27.763 INFO:teuthology.orchestra.run.smithi042.stdout:average_utilization 0.0010 2024-09-18T01:00:27.763 INFO:teuthology.orchestra.run.smithi042.stdout:overload_utilization 0.0011 2024-09-18T01:00:27.763 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 weight 0.8141 -> 0.6627 2024-09-18T01:00:27.763 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 weight 0.7792 -> 0.6828 2024-09-18T01:00:27.763 INFO:teuthology.orchestra.run.smithi042.stderr:no change 2024-09-18T01:00:27.814 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:27.812+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:27.830 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:27.830 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:27.830 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:27.831 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:27.843 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:00:27.915 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:27.916+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:27.971 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:27.971 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:27.985 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:00:28.016 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:28.016+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:28.117 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:28.116+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:28.120 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 159.054890914, 2024-09-18T01:00:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T01:00:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T01:00:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T01:00:28.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T01:00:28.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.127 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.162357095000004, 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T01:00:28.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T01:00:28.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.413290305000004, 2024-09-18T01:00:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T01:00:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:28.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T01:00:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T01:00:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T01:00:28.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T01:00:28.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.402915848999996, 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T01:00:28.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T01:00:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T01:00:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T01:00:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T01:00:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T01:00:28.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T01:00:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.402843167, 2024-09-18T01:00:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T01:00:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T01:00:28.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T01:00:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T01:00:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T01:00:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T01:00:28.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T01:00:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T01:00:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.149 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.402812300999997, 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:00:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T01:00:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:00:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T01:00:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T01:00:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.154 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.402759527000001, 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T01:00:28.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:28.159 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.402733361000003, 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:28.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.402708206, 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T01:00:28.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T01:00:28.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.402284894999994, 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:28.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.173 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.402246516999995, 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.174 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T01:00:28.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T01:00:28.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:28.178 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.402221752000003, 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:28.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T01:00:28.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T01:00:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T01:00:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T01:00:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T01:00:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.395426987999997, 2024-09-18T01:00:28.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T01:00:28.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.395401387999996, 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T01:00:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T01:00:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T01:00:28.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T01:00:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T01:00:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T01:00:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T01:00:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T01:00:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T01:00:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T01:00:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T01:00:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.395378045000001, 2024-09-18T01:00:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T01:00:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.194 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T01:00:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:00:28.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T01:00:28.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.395355605999995, 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T01:00:28.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T01:00:28.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.395334833000007, 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T01:00:28.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T01:00:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T01:00:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T01:00:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T01:00:28.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T01:00:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T01:00:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T01:00:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T01:00:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T01:00:28.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T01:00:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T01:00:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.209 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.395313806999994, 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T01:00:28.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T01:00:28.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T01:00:28.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T01:00:28.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T01:00:28.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.214 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.395273224999997, 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T01:00:28.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.395239126000007, 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T01:00:28.219 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:28.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:28.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:28.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:00:28.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T01:00:28.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T01:00:28.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T01:00:28.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.224 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.224 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.224 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:28.226 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:00:28.226 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:28.224+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:28.326 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:28.324+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:28.356 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:28.356 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:28.371 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:00:28.427 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:28.424+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:28.497 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:28.497 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:28.497 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:28.497 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:28.497 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:28.511 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:00:28.529 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:28.528+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:28.629 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:28.624+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:28.678 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:28.678 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 185.385256424, 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T01:00:28.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T01:00:28.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:28.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T01:00:28.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:28.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T01:00:28.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T01:00:28.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T01:00:28.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.18889393399999, 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.690 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.188798607, 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T01:00:28.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.18877316800001, 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T01:00:28.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T01:00:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T01:00:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T01:00:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T01:00:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T01:00:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.188750645, 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.700 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T01:00:28.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.18872720499999, 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T01:00:28.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T01:00:28.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.188703395, 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T01:00:28.709 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:00:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T01:00:28.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.18868103, 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T01:00:28.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.18865441299999, 2024-09-18T01:00:28.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.184436334, 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T01:00:28.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.184409993, 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.731 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.18438732300001, 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.18436894999999, 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.736 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:28.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T01:00:28.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:28.740 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.18434916199999, 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:28.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T01:00:28.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T01:00:28.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 160.18432229199999, 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:28.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:00:28.749 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 113.10813182299999, 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T01:00:28.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 71.697066727000006, 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T01:00:28.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T01:00:28.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T01:00:28.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T01:00:28.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.762 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:140635 3.d 3:b08054e4:::rbd_object_map.117ced55205b.0000000000000023:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-18T01:00:28.762 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:28.764 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 57.335953332999999, 2024-09-18T01:00:28.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86222047999999996, 2024-09-18T01:00:28.764 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 140635 2024-09-18T01:00:28.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.764 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338625+0000", 2024-09-18T01:00:28.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6199999999999999e-06 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338634+0000", 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9099999999999994e-06 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338657+0000", 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3672999999999998e-05 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338675+0000", 2024-09-18T01:00:28.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7589999999999999e-05 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338767+0000", 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1896000000000002e-05 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338776+0000", 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.38e-06 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200213+0000", 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86143668200000001 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200432+0000", 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021884199999999999 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200844+0000", 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000411887 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:155664 3.d 3:b496760f:::rbd_data.117c1d78ece1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.768650229000002, 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046053647000000003, 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 155664 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905927+0000", 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.175e-06 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909267+0000", 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003339266 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909287+0000", 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605e-05 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909297+0000", 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3139999999999993e-06 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909363+0000", 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6437000000000001e-05 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909483+0000", 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119861 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909868+0000", 2024-09-18T01:00:28.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038564099999999999 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.917828+0000", 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079592570000000008 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951788+0000", 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033960314999999998 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951895+0000", 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107316 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951980+0000", 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4460000000000001e-05 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:161321 3.d 3:b70a3213:::rbd_header.117c59271281:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e151)", 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:28.774 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.571011330000001, 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.84105173099999997, 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 161321 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103567+0000", 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.553e-06 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103576+0000", 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4099999999999997e-06 2024-09-18T01:00:28.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103601+0000", 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4833000000000001e-05 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103620+0000", 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9409999999999999e-05 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103789+0000", 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016871599999999999 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103807+0000", 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8502999999999999e-05 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.943761+0000", 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.83995325200000004 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.944023+0000", 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026232799999999999 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:28.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:28.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.944617+0000", 2024-09-18T01:00:28.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059372600000000004 2024-09-18T01:00:28.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.779 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:28.779 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:28.779 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:28.779 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:00:28.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:00:28.782 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:28.776+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:28.882 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:28.880+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:28.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:00:28.958 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:28.958 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:28.958 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:28.958 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:28.979 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:00:28.982 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:28.980+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:28.996 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:28.997 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:29.016 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:29.034 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:29.035 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:29.056 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:29.077 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:29.077 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:29.083 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:29.080+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:29.097 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:29.097 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:29.118 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:29.118 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:29.134 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:29.134 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:29.137 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:29.137 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:29.150 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:00:29.183 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:29.180+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:29.284 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:29.280+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:29.316 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:29.317 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:29.331 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:00:29.418 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:29.416+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:29.486 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:29.484+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:29.498 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:29.511 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:00:29.598 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:29.596+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:29.665 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:29.665 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:29.678 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:00:29.690 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:29.688+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:29.791 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:29.788+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:29.848 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:29.848 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:29.862 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:00:29.892 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:29.888+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:29.992 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:29.992+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:29.996 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:29.997 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:30.011 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:00:30.063 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T01:00:30.093 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:30.088+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:30.184 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:30.184 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:30.194 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:30.192+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:30.199 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:00:30.295 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:30.292+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:30.328 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:30.328 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:30.342 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:00:30.395 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:30.392+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:30.496 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:30.496+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:30.509 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:30.509 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:30.523 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:00:30.608 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:30.608+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:30.681 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:30.681 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:30.694 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:00:30.698 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:30.696+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:30.799 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:30.796+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:30.862 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:30.863 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:30.876 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:00:30.920 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:30.920+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:31.001 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:31.000+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:31.043 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:31.043 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:31.057 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:00:31.102 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:31.100+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:31.159 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:00:31.206 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:31.204+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:31.226 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:31.227 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:31.241 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:00:31.303 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:31.300+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:31.318 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:31.318 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:31.334 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:31.335 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:31.351 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:31.351 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:31.361 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:31.362 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:31.369 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:31.370 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:31.375 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:00:31.389 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:31.389 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:31.406 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:31.407 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:31.420 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:31.420+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:31.425 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:31.425 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:31.443 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:31.444 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:31.494 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:31.494 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:31.494 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:31.494 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:31.505 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:31.504+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:31.508 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:00:31.606 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:31.604+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:31.673 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:31.674 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:31.674 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-18T01:00:31.674 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-18T01:00:31.674 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:31.687 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:00:31.732 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:31.732+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:31.808 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:31.808+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:31.861 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:31.861 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-18T01:00:31.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-18T01:00:31.861 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4043 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 166.85621181900001, 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037668506999999997, 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4043 2024-09-18T01:00:31.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.863 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:31.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003062+0000", 2024-09-18T01:00:31.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003063+0000", 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1999999999999999e-06 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003068+0000", 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3170000000000001e-06 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003087+0000", 2024-09-18T01:00:31.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9219999999999999e-05 2024-09-18T01:00:31.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003101+0000", 2024-09-18T01:00:31.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3823e-05 2024-09-18T01:00:31.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040185+0000", 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037084142 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040324+0000", 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000139133 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040730+0000", 2024-09-18T01:00:31.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040567199999999999 2024-09-18T01:00:31.867 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.867 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.867 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.867 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4044 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:31.867 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:31.867 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 166.856153562, 2024-09-18T01:00:31.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038211394000000003, 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4044 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:31.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003120+0000", 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003121+0000", 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.243e-06 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003127+0000", 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6200000000000004e-06 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003145+0000", 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.789e-05 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003161+0000", 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6169000000000001e-05 2024-09-18T01:00:31.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.040854+0000", 2024-09-18T01:00:31.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037693292000000003 2024-09-18T01:00:31.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041005+0000", 2024-09-18T01:00:31.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015137899999999999 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041331+0000", 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032580100000000002 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4046 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:31.872 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 166.85609512299999, 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038801978000000001, 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4046 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003178+0000", 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003179+0000", 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1769999999999999e-06 2024-09-18T01:00:31.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003185+0000", 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2900000000000002e-06 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003205+0000", 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0333e-05 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003220+0000", 2024-09-18T01:00:31.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4552999999999999e-05 2024-09-18T01:00:31.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041466+0000", 2024-09-18T01:00:31.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038246352999999997 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041650+0000", 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184191 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.041980+0000", 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033008100000000003 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.878 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4048 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:31.878 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:31.878 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 166.85603390599999, 2024-09-18T01:00:31.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039309095000000002, 2024-09-18T01:00:31.878 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.878 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.878 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.878 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.878 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.878 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4048 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003240+0000", 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003241+0000", 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7430000000000001e-06 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003247+0000", 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8200000000000002e-06 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003266+0000", 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8906000000000002e-05 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003280+0000", 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4236999999999999e-05 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042046+0000", 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038765388999999997 2024-09-18T01:00:31.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042233+0000", 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018753699999999999 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042549+0000", 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031546299999999998 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.882 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4049 3.b 3:d6b1830c:::rbd_data.117c7eee262f.0000000000000019:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 166.855943491, 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040129274, 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4049 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003330+0000", 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003332+0000", 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.584e-06 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003339+0000", 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2799999999999998e-06 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003361+0000", 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1759999999999998e-05 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003378+0000", 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7390000000000001e-05 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042652+0000", 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039274130999999997 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.042902+0000", 2024-09-18T01:00:31.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024950999999999999 2024-09-18T01:00:31.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043459+0000", 2024-09-18T01:00:31.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055761899999999997 2024-09-18T01:00:31.887 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.887 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4053 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 166.85587255799999, 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040554534000000003, 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.888 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4053 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003401+0000", 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003402+0000", 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003408+0000", 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9599999999999997e-06 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003427+0000", 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8732999999999998e-05 2024-09-18T01:00:31.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003441+0000", 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3900000000000001e-05 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043566+0000", 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040125474000000001 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043735+0000", 2024-09-18T01:00:31.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168283 2024-09-18T01:00:31.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.043955+0000", 2024-09-18T01:00:31.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000220757 2024-09-18T01:00:31.892 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.892 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.892 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4055 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 166.85581295200001, 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041069395000000002, 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4055 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.893 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003460+0000", 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003462+0000", 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.13e-06 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003467+0000", 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4700000000000001e-06 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003486+0000", 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8624e-05 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003500+0000", 2024-09-18T01:00:31.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3825999999999999e-05 2024-09-18T01:00:31.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044060+0000", 2024-09-18T01:00:31.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040560710999999999 2024-09-18T01:00:31.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044281+0000", 2024-09-18T01:00:31.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022123800000000001 2024-09-18T01:00:31.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044530+0000", 2024-09-18T01:00:31.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024839600000000001 2024-09-18T01:00:31.897 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.897 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.897 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4056 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 166.85575358200001, 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041329363000000001, 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4056 2024-09-18T01:00:31.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.899 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:31.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003520+0000", 2024-09-18T01:00:31.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003521+0000", 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.257e-06 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003526+0000", 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2970000000000003e-06 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003545+0000", 2024-09-18T01:00:31.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9029e-05 2024-09-18T01:00:31.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003564+0000", 2024-09-18T01:00:31.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8499999999999999e-05 2024-09-18T01:00:31.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044611+0000", 2024-09-18T01:00:31.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041047291999999999 2024-09-18T01:00:31.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044718+0000", 2024-09-18T01:00:31.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000106587 2024-09-18T01:00:31.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044849+0000", 2024-09-18T01:00:31.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013140100000000001 2024-09-18T01:00:31.903 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.903 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.903 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.903 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4059 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:31.903 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:31.903 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 166.85569179300001, 2024-09-18T01:00:31.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041763738000000002, 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4059 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:31.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003582+0000", 2024-09-18T01:00:31.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003583+0000", 2024-09-18T01:00:31.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.24e-06 2024-09-18T01:00:31.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003600+0000", 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7167e-05 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003625+0000", 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5060000000000001e-05 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003641+0000", 2024-09-18T01:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5460000000000001e-05 2024-09-18T01:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.044908+0000", 2024-09-18T01:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041267469000000001 2024-09-18T01:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045107+0000", 2024-09-18T01:00:31.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019927599999999999 2024-09-18T01:00:31.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045345+0000", 2024-09-18T01:00:31.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023806599999999999 2024-09-18T01:00:31.908 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.908 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.909 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.909 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:4060 3.b 3:d0a3e06c:::rbd_data.117c7eee262f.000000000000001d:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:00:31.909 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:31.909 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 166.85555897, 2024-09-18T01:00:31.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041888386, 2024-09-18T01:00:31.909 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4060 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:31.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003714+0000", 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003716+0000", 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5629999999999999e-06 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003723+0000", 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6540000000000002e-06 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003760+0000", 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6986000000000001e-05 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.003775+0000", 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5173e-05 2024-09-18T01:00:31.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045373+0000", 2024-09-18T01:00:31.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041597702 2024-09-18T01:00:31.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045465+0000", 2024-09-18T01:00:31.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2365000000000001e-05 2024-09-18T01:00:31.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:45.045603+0000", 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013794299999999999 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8443 3.b 3:d466d430:::rbd_data.117c6973391c.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 163.30034149299999, 2024-09-18T01:00:31.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037931078999999999, 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8443 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558932+0000", 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558933+0000", 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2900000000000002e-07 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558945+0000", 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1878999999999999e-05 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558955+0000", 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0172e-05 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.558961+0000", 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8610000000000003e-06 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595027+0000", 2024-09-18T01:00:31.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036066066000000001 2024-09-18T01:00:31.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595155+0000", 2024-09-18T01:00:31.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012785100000000001 2024-09-18T01:00:31.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:31.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.595341+0000", 2024-09-18T01:00:31.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000185916 2024-09-18T01:00:31.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:31.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596781+0000", 2024-09-18T01:00:31.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001440263 2024-09-18T01:00:31.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:31.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596810+0000", 2024-09-18T01:00:31.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8960999999999999e-05 2024-09-18T01:00:31.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:31.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596840+0000", 2024-09-18T01:00:31.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0281e-05 2024-09-18T01:00:31.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596863+0000", 2024-09-18T01:00:31.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3e-05 2024-09-18T01:00:31.921 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.921 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.921 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.921 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8461 3.b 3:d735a6a1:::rbd_data.117ce245106a.0000000000000003:5 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:31.921 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:31.921 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 163.29984136900001, 2024-09-18T01:00:31.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037140676999999997, 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8461 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559432+0000", 2024-09-18T01:00:31.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559433+0000", 2024-09-18T01:00:31.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3099999999999997e-07 2024-09-18T01:00:31.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559436+0000", 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.05e-06 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559444+0000", 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.695e-06 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559453+0000", 2024-09-18T01:00:31.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2809999999999997e-06 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596355+0000", 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036902138000000001 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596496+0000", 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014116500000000001 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596573+0000", 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6617000000000002e-05 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8462 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 163.299810861, 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037320524000000001, 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.926 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.927 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.927 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.927 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.927 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.927 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8462 2024-09-18T01:00:31.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.927 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559463+0000", 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0100000000000005e-07 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559466+0000", 2024-09-18T01:00:31.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1429999999999998e-06 2024-09-18T01:00:31.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559474+0000", 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5320000000000001e-06 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559479+0000", 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596609+0000", 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037129926000000001 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596700+0000", 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0797999999999994e-05 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596783+0000", 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3441000000000003e-05 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.931 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8463 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 163.29978619600001, 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037626693000000003, 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8463 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559487+0000", 2024-09-18T01:00:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559488+0000", 2024-09-18T01:00:31.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:00:31.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559490+0000", 2024-09-18T01:00:31.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.74e-06 2024-09-18T01:00:31.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559497+0000", 2024-09-18T01:00:31.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9739999999999996e-06 2024-09-18T01:00:31.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559503+0000", 2024-09-18T01:00:31.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1830000000000002e-06 2024-09-18T01:00:31.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.596923+0000", 2024-09-18T01:00:31.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037420163999999999 2024-09-18T01:00:31.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597017+0000", 2024-09-18T01:00:31.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4000999999999997e-05 2024-09-18T01:00:31.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597114+0000", 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7201000000000007e-05 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8464 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 163.299757664, 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037827195000000001, 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.937 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8464 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559516+0000", 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559519+0000", 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9280000000000002e-06 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559527+0000", 2024-09-18T01:00:31.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5669999999999997e-06 2024-09-18T01:00:31.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559533+0000", 2024-09-18T01:00:31.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-18T01:00:31.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597159+0000", 2024-09-18T01:00:31.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037625955000000003 2024-09-18T01:00:31.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597270+0000", 2024-09-18T01:00:31.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110505 2024-09-18T01:00:31.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597343+0000", 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3443999999999997e-05 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8465 3.b 3:d5bafad9:::rbd_data.117ce245106a.0000000000000004:5 [read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 163.29942437299999, 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037624488999999997, 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.942 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8465 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559849+0000", 2024-09-18T01:00:31.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559850+0000", 2024-09-18T01:00:31.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7000000000000004e-07 2024-09-18T01:00:31.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559854+0000", 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2819999999999998e-06 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559865+0000", 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1308000000000001e-05 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559873+0000", 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5120000000000002e-06 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597379+0000", 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037506534000000001 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597429+0000", 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0008e-05 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597474+0000", 2024-09-18T01:00:31.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4075000000000001e-05 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8476 3.b 3:d045dd73:::rbd_data.117c6973391c.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 163.299382522, 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038949009, 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:31.947 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8476 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559891+0000", 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559892+0000", 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9100000000000003e-07 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559896+0000", 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.963e-06 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559905+0000", 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.054e-06 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.559921+0000", 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6365e-05 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597503+0000", 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037582338999999999 2024-09-18T01:00:31.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597590+0000", 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6664999999999998e-05 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597706+0000", 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116099 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598755+0000", 2024-09-18T01:00:31.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010485799999999999 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598791+0000", 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6675999999999999e-05 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598820+0000", 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9113999999999999e-05 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598840+0000", 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9463e-05 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:8479 3.b 3:d0cfb88b:::rbd_data.117ce245106a.000000000000000e:5 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 163.29834502599999, 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.037190297999999997, 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-18T01:00:31.953 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 8479 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560928+0000", 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560930+0000", 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560936+0000", 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6850000000000004e-06 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560950+0000", 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3856000000000001e-05 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.560959+0000", 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5050000000000007e-06 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597935+0000", 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036975903999999997 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.597996+0000", 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1112999999999996e-05 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:57:48.598119+0000", 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012292499999999999 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.957 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:73104 3.b 3:d043daa9:::rbd_data.117c304cc916.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 112.646057021, 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047616591, 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 73104 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213216+0000", 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213219+0000", 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.227e-06 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213230+0000", 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0973e-05 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213261+0000", 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1272999999999997e-05 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213276+0000", 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4956999999999999e-05 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213423+0000", 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014697199999999999 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.213580+0000", 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015697199999999999 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.221932+0000", 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083520209999999994 2024-09-18T01:00:31.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260624+0000", 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038691857000000003 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260699+0000", 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5430000000000001e-05 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260773+0000", 2024-09-18T01:00:31.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3947000000000002e-05 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:58:39.260833+0000", 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9962e-05 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4476.0:138250 3.c 3:32730bfd:::rbd_data.117c1c3044f5.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.100359213000004, 2024-09-18T01:00:31.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.060687680000000001, 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4476", 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 138250 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758914+0000", 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758917+0000", 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5550000000000001e-06 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758925+0000", 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5900000000000008e-06 2024-09-18T01:00:31.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758947+0000", 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1197000000000001e-05 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.758957+0000", 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0137e-05 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759094+0000", 2024-09-18T01:00:31.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013775300000000001 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.759246+0000", 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015127900000000001 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.770839+0000", 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011593538 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-18T01:00:31.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819413+0000", 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.048573660999999997 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819467+0000", 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3795999999999998e-05 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819529+0000", 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2675999999999994e-05 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-18T01:00:31.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-18T01:00:31.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-18T00:59:21.819602+0000", 2024-09-18T01:00:31.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2497999999999996e-05 2024-09-18T01:00:31.969 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.969 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.969 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.969 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-18T01:00:31.969 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-18T01:00:31.969 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:31.970 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:31.968+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:31.971 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:00:32.072 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:32.072+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:32.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:00:32.115 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:32.115 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:32.115 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:32.131 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:00:32.173 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:32.172+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:32.274 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:32.272+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:32.303 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:32.303 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:32.303 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:32.304 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:32.304 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:32.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:00:32.375 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:32.372+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:32.475 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:32.476+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:32.498 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:32.498 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:32.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:32.498 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:32.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.498 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 163.42976902800001, 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T01:00:32.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T01:00:32.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T01:00:32.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.504 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.537235209000002, 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.505 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T01:00:32.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.506 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:32.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.510 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.788168419000002, 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T01:00:32.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.777793962999993, 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:32.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.520 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.520 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:32.520 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.777721280999998, 2024-09-18T01:00:32.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T01:00:32.520 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T01:00:32.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T01:00:32.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T01:00:32.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T01:00:32.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T01:00:32.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T01:00:32.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T01:00:32.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T01:00:32.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T01:00:32.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T01:00:32.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T01:00:32.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T01:00:32.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T01:00:32.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T01:00:32.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.525 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.777690414999995, 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T01:00:32.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:32.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T01:00:32.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.777637640999998, 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T01:00:32.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.532 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:32.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:32.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T01:00:32.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T01:00:32.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T01:00:32.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.777611475, 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T01:00:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:32.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T01:00:32.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.777586319999997, 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T01:00:32.541 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T01:00:32.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.542 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T01:00:32.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.777163009000006, 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:32.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T01:00:32.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T01:00:32.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T01:00:32.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T01:00:32.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T01:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T01:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T01:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T01:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T01:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T01:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T01:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T01:00:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T01:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T01:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.777124631000007, 2024-09-18T01:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T01:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T01:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T01:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T01:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T01:00:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T01:00:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T01:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T01:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T01:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T01:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T01:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T01:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T01:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T01:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T01:00:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.777099866, 2024-09-18T01:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T01:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T01:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T01:00:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.770305101999995, 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T01:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T01:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T01:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T01:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T01:00:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.770279501999994, 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T01:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T01:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T01:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T01:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.770256158999999, 2024-09-18T01:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T01:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T01:00:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.770233719999993, 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:00:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T01:00:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.770212947000005, 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T01:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T01:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T01:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T01:00:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T01:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T01:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T01:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T01:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.770191921000006, 2024-09-18T01:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T01:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T01:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T01:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T01:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T01:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T01:00:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T01:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T01:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T01:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T01:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T01:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T01:00:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T01:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T01:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.770151338999995, 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T01:00:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T01:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T01:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T01:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T01:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T01:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T01:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T01:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T01:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 83.770117240000005, 2024-09-18T01:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T01:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T01:00:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T01:00:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T01:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T01:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T01:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:32.610 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:32.612 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:00:32.612 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:32.608+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:32.713 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:32.708+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:32.780 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6, 7, 3] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:00:32.781 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:00:32.781 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:00:32.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:00:32.785 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:32.785 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:32.785 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:32.801 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:00:32.846 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:32.844+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:32.915 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:32.912+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:32.974 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:32.989 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:00:33.015 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:33.016+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:33.073 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T01:00:33.104 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:00:33.104 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:33.104 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 175, 2024-09-18T01:00:33.104 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "82a96d76-ba83-49e2-a985-84604bd599ba", 2024-09-18T01:00:33.104 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-18T00:57:07.138732+0000", 2024-09-18T01:00:33.104 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-18T01:00:33.059233+0000", 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-18T00:57:20.258605+0000", 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-18T01:00:14.958457+0000", 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5799936, 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 5, 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "jewel", 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:21.962521+0000", 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-18T01:00:33.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-18T01:00:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "19", 2024-09-18T01:00:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T01:00:33.109 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T01:00:33.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-18T01:00:33.111 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 8, 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 8, 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.25, 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:00:33.112 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:25.721733+0000", 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T01:00:33.113 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T01:00:33.114 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "23", 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:00:33.115 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 23, 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:00:33.116 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:00:33.117 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 4, 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 4, 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.75, 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 3, 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 3, 2024-09-18T01:00:33.118 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-18T00:57:35.599413+0000", 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-18T01:00:33.119 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:00:33.120 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:00:33.120 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:00:33.120 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:00:33.120 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-18T01:00:33.120 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-18T01:00:33.120 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:00:33.120 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-18T01:00:33.120 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-18T01:00:33.121 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-18T01:00:33.121 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-18T01:00:33.121 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-18T01:00:33.121 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-18T01:00:33.121 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-18T01:00:33.121 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "175", 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 45, 2024-09-18T01:00:33.122 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 175, 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:00:33.123 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:00:33.124 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 1.75, 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 1.75, 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.75, 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.309999942779541, 2024-09-18T01:00:33.125 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.309999942779541, 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "3f359ce5-6f6e-4161-8554-8ac028347274", 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T01:00:33.126 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 95, 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 199763443 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/199763443", 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/199763443", 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/199763443", 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/199763443", 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:33.131 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "48687611-1556-4b6d-86af-7872494a91d7", 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 164, 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:33.132 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 345149800 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/345149800", 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/345149800", 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/345149800", 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/345149800", 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "9e6b1f90-ce9b-46d4-92fe-d457f7e415d5", 2024-09-18T01:00:33.137 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 84, 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.138 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.139 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 431239931 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/431239931", 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/431239931", 2024-09-18T01:00:33.142 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/431239931", 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/431239931", 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5", 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0.8140716552734375, 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:33.143 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 169, 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1775474180 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/1775474180", 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/1775474180", 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/1775474180", 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/1775474180", 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "802ccbeb-7351-4b38-832a-695117c7e493", 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0.7791595458984375, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 172, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:33.149 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6808", 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6809", 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6810", 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6811", 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.151 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6814", 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6815", 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.152 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6812", 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6813", 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4198401207 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.153 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6809/4198401207", 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6811/4198401207", 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6815/4198401207", 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6813/4198401207", 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "d958b3da-efa1-4d86-b81d-e78874441437", 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:33.154 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 172, 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6824", 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6825", 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6826", 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:33.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6827", 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6830", 2024-09-18T01:00:33.157 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6831", 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6828", 2024-09-18T01:00:33.158 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:33.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.159 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.159 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6829", 2024-09-18T01:00:33.159 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2599825823 2024-09-18T01:00:33.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.159 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.160 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6825/2599825823", 2024-09-18T01:00:33.160 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6827/2599825823", 2024-09-18T01:00:33.160 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6831/2599825823", 2024-09-18T01:00:33.160 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6829/2599825823", 2024-09-18T01:00:33.160 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:33.160 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:33.160 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:33.160 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.160 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "51b3cea8-c686-4fd3-b3f4-ba936cf26b64", 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 172, 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6800", 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6801", 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:33.162 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6802", 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6803", 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:33.163 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6806", 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6807", 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.164 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6804", 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6805", 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3550656451 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.165 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6801/3550656451", 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6803/3550656451", 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6807/3550656451", 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6805/3550656451", 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "b02ead1c-837a-4b07-9ad1-4e903ed5bc4e", 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-18T01:00:33.166 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 161, 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6816", 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:33.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6817", 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6818", 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:33.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6819", 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6822", 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:33.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6823", 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6820", 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:33.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.195:6821", 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2993352359 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.195:6817/2993352359", 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.195:6819/2993352359", 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.195:6823/2993352359", 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.195:6821/2993352359", 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-18T01:00:33.171 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.896477+0000", 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.941756+0000", 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:33.173 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.560180+0000", 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.077394+0000", 2024-09-18T01:00:33.174 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.996264+0000", 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.175 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.511686+0000", 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:33.176 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:17.411051+0000", 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-18T01:00:33.177 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-18T00:57:16.888092+0000", 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-18T01:00:33.178 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [ 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 14, 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 6 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [ 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 14, 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-18T01:00:33.180 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-18T01:00:33.181 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-18T01:00:33.181 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-18T01:00:33.181 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-18T01:00:33.181 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-18T01:00:33.181 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:00:33.181 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:00:33.181 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-18T01:00:33.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.181 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:33.184 INFO:tasks.thrashosds.thrasher:j is {'epoch': 175, 'fsid': '82a96d76-ba83-49e2-a985-84604bd599ba', 'created': '2024-09-18T00:57:07.138732+0000', 'modified': '2024-09-18T01:00:33.059233+0000', 'last_up_change': '2024-09-18T00:57:20.258605+0000', 'last_in_change': '2024-09-18T01:00:14.958457+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T00:57:21.962521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T00:57:25.721733+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.75, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T00:57:35.599413+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 175, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 0.75, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3f359ce5-6f6e-4161-8554-8ac028347274', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 199763443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 199763443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 199763443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 199763443}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 199763443}]}, 'public_addr': '172.21.15.42:6801/199763443', 'cluster_addr': '172.21.15.42:6803/199763443', 'heartbeat_back_addr': '172.21.15.42:6807/199763443', 'heartbeat_front_addr': '172.21.15.42:6805/199763443', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '48687611-1556-4b6d-86af-7872494a91d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 345149800}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 345149800}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 345149800}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 345149800}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 345149800}]}, 'public_addr': '172.21.15.42:6809/345149800', 'cluster_addr': '172.21.15.42:6811/345149800', 'heartbeat_back_addr': '172.21.15.42:6815/345149800', 'heartbeat_front_addr': '172.21.15.42:6813/345149800', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9e6b1f90-ce9b-46d4-92fe-d457f7e415d5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 431239931}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 431239931}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 431239931}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 431239931}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 431239931}]}, 'public_addr': '172.21.15.42:6825/431239931', 'cluster_addr': '172.21.15.42:6827/431239931', 'heartbeat_back_addr': '172.21.15.42:6831/431239931', 'heartbeat_front_addr': '172.21.15.42:6829/431239931', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5', 'up': 1, 'in': 1, 'weight': 0.8140716552734375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 169, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 1775474180}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 1775474180}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 1775474180}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 1775474180}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 1775474180}]}, 'public_addr': '172.21.15.42:6817/1775474180', 'cluster_addr': '172.21.15.42:6819/1775474180', 'heartbeat_back_addr': '172.21.15.42:6823/1775474180', 'heartbeat_front_addr': '172.21.15.42:6821/1775474180', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '802ccbeb-7351-4b38-832a-695117c7e493', 'up': 1, 'in': 1, 'weight': 0.7791595458984375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 172, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6809', 'nonce': 4198401207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6811', 'nonce': 4198401207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6815', 'nonce': 4198401207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 4198401207}, {'type': 'v1', 'addr': '172.21.15.195:6813', 'nonce': 4198401207}]}, 'public_addr': '172.21.15.195:6809/4198401207', 'cluster_addr': '172.21.15.195:6811/4198401207', 'heartbeat_back_addr': '172.21.15.195:6815/4198401207', 'heartbeat_front_addr': '172.21.15.195:6813/4198401207', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd958b3da-efa1-4d86-b81d-e78874441437', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 172, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6824', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6825', 'nonce': 2599825823}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6826', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6827', 'nonce': 2599825823}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6830', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6831', 'nonce': 2599825823}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6828', 'nonce': 2599825823}, {'type': 'v1', 'addr': '172.21.15.195:6829', 'nonce': 2599825823}]}, 'public_addr': '172.21.15.195:6825/2599825823', 'cluster_addr': '172.21.15.195:6827/2599825823', 'heartbeat_back_addr': '172.21.15.195:6831/2599825823', 'heartbeat_front_addr': '172.21.15.195:6829/2599825823', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '51b3cea8-c686-4fd3-b3f4-ba936cf26b64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 172, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6800', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6801', 'nonce': 3550656451}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6803', 'nonce': 3550656451}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6807', 'nonce': 3550656451}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 3550656451}, {'type': 'v1', 'addr': '172.21.15.195:6805', 'nonce': 3550656451}]}, 'public_addr': '172.21.15.195:6801/3550656451', 'cluster_addr': '172.21.15.195:6803/3550656451', 'heartbeat_back_addr': '172.21.15.195:6807/3550656451', 'heartbeat_front_addr': '172.21.15.195:6805/3550656451', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b02ead1c-837a-4b07-9ad1-4e903ed5bc4e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 161, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6817', 'nonce': 2993352359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6818', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6819', 'nonce': 2993352359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6822', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6823', 'nonce': 2993352359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6820', 'nonce': 2993352359}, {'type': 'v1', 'addr': '172.21.15.195:6821', 'nonce': 2993352359}]}, 'public_addr': '172.21.15.195:6817/2993352359', 'cluster_addr': '172.21.15.195:6819/2993352359', 'heartbeat_back_addr': '172.21.15.195:6823/2993352359', 'heartbeat_front_addr': '172.21.15.195:6821/2993352359', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.896477+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.941756+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.560180+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.077394+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.996264+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.511686+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:17.411051+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:57:16.888092+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 14, 'length': 6}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 14, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:00:33.185 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 189.87711487300001, 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T01:00:33.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T01:00:33.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:33.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T01:00:33.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T01:00:33.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:33.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T01:00:33.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T01:00:33.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:33.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T01:00:33.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T01:00:33.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:33.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T01:00:33.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T01:00:33.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T01:00:33.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T01:00:33.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.194 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.194 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.195 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:33.195 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.680752383, 2024-09-18T01:00:33.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T01:00:33.195 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.196 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.196 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.196 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T01:00:33.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.196 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:33.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:33.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T01:00:33.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T01:00:33.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T01:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T01:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T01:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T01:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T01:00:33.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.680657056, 2024-09-18T01:00:33.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:33.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T01:00:33.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.68063161699999, 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:33.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T01:00:33.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.210 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.680609094, 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T01:00:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T01:00:33.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.680585654, 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.215 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T01:00:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T01:00:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T01:00:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T01:00:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T01:00:33.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T01:00:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T01:00:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T01:00:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.68056184400001, 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T01:00:33.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.680539479, 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.225 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T01:00:33.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.680512862, 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.233 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.676294783, 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T01:00:33.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T01:00:33.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.67626844200001, 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.242 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.67624577199999, 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T01:00:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.676227399, 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.247 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T01:00:33.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T01:00:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T01:00:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T01:00:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T01:00:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.251 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:33.252 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.676207611, 2024-09-18T01:00:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T01:00:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T01:00:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T01:00:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 164.676180741, 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T01:00:33.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T01:00:33.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.599990272, 2024-09-18T01:00:33.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T01:00:33.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T01:00:33.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 76.188925175999998, 2024-09-18T01:00:33.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T01:00:33.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T01:00:33.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T01:00:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T01:00:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:140635 3.d 3:b08054e4:::rbd_object_map.117ced55205b.0000000000000023:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-18T01:00:33.275 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.827811781999998, 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86222047999999996, 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 140635 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338625+0000", 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6199999999999999e-06 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338634+0000", 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9099999999999994e-06 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338657+0000", 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3672999999999998e-05 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338675+0000", 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7589999999999999e-05 2024-09-18T01:00:33.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338767+0000", 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1896000000000002e-05 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338776+0000", 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.38e-06 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200213+0000", 2024-09-18T01:00:33.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86143668200000001 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200432+0000", 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021884199999999999 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200844+0000", 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000411887 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:155664 3.d 3:b496760f:::rbd_data.117c1d78ece1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-18T01:00:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.260508678000001, 2024-09-18T01:00:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046053647000000003, 2024-09-18T01:00:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 155664 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905927+0000", 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.175e-06 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909267+0000", 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003339266 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909287+0000", 2024-09-18T01:00:33.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605e-05 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909297+0000", 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3139999999999993e-06 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909363+0000", 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6437000000000001e-05 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909483+0000", 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119861 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909868+0000", 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038564099999999999 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.917828+0000", 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079592570000000008 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951788+0000", 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033960314999999998 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951895+0000", 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107316 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951980+0000", 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4460000000000001e-05 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.286 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:161321 3.d 3:b70a3213:::rbd_header.117c59271281:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e151)", 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.062869779, 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.84105173099999997, 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:33.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 161321 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103567+0000", 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.553e-06 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103576+0000", 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4099999999999997e-06 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103601+0000", 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4833000000000001e-05 2024-09-18T01:00:33.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103620+0000", 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9409999999999999e-05 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103789+0000", 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016871599999999999 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103807+0000", 2024-09-18T01:00:33.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8502999999999999e-05 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.943761+0000", 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.83995325200000004 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.944023+0000", 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026232799999999999 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.944617+0000", 2024-09-18T01:00:33.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059372600000000004 2024-09-18T01:00:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:33.292 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:33.292 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:33.293 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:00:33.293 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:33.180+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:33.297 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:33.292+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:33.396 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:33.396+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:33.410 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:33.410 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:33.410 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:33.410 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:33.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:00:33.461 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:00:33.497 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:33.496+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:33.535 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:33.535 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:33.536 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:33.536 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:33.549 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:00:33.597 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:33.596+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:33.599 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:00:33.600 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T01:00:33.605 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:33.605 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:33.618 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-18T01:00:33.069142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":339779179,"num_objects":1012,"num_object_clones":140,"num_object_copies":2024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1012,"num_whiteouts":602,"num_read":217258,"num_read_kb":1064438,"num_write":21696,"num_write_kb":690823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9854,"num_bytes_recovered":693846218,"num_keys_recovered":1158,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18603,"ondisk_log_size":18603,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":529032,"kb_used_data":241844,"kb_used_omap":9,"kb_used_meta":286966,"kb_avail":565702008,"statfs":{"total":579820584960,"available":579278856192,"internally_reserved":0,"allocated":247648256,"data_stored":254173990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":293853862},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-75115067,"num_objects":-217,"num_object_clones":-82,"num_object_copies":-434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-217,"num_whiteouts":-79,"num_read":806,"num_read_kb":692,"num_write":159,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":4062059,"num_keys_recovered":12,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.139866"},"pg_stats":[{"pgid":"3.1f","version":"172'609","reported_seq":4061,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.218681+0000","last_change":"2024-09-18T01:00:30.076068+0000","last_active":"2024-09-18T01:00:30.218681+0000","last_peered":"2024-09-18T01:00:30.218681+0000","last_clean":"2024-09-18T01:00:30.218681+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T01:00:30.218681+0000","last_undegraded":"2024-09-18T01:00:30.218681+0000","last_fullsized":"2024-09-18T01:00:30.218681+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:57.331138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057633090000000003,"stat_sum":{"num_bytes":1474576,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":19,"num_read":6433,"num_read_kb":23163,"num_write":610,"num_write_kb":11006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.1e","version":"172'634","reported_seq":5336,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.206184+0000","last_change":"2024-09-18T01:00:30.080693+0000","last_active":"2024-09-18T01:00:30.206184+0000","last_peered":"2024-09-18T01:00:30.206184+0000","last_clean":"2024-09-18T01:00:30.206184+0000","last_became_active":"2024-09-18T01:00:23.947401+0000","last_became_peered":"2024-09-18T01:00:23.947401+0000","last_unstale":"2024-09-18T01:00:30.206184+0000","last_undegraded":"2024-09-18T01:00:30.206184+0000","last_fullsized":"2024-09-18T01:00:30.206184+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:08:30.009124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072113790000000004,"stat_sum":{"num_bytes":5035983,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":25,"num_read":4820,"num_read_kb":36391,"num_write":697,"num_write_kb":11559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":30825045,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.1d","version":"172'766","reported_seq":12529,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.208906+0000","last_change":"2024-09-18T01:00:30.073358+0000","last_active":"2024-09-18T01:00:30.208906+0000","last_peered":"2024-09-18T01:00:30.208906+0000","last_clean":"2024-09-18T01:00:30.208906+0000","last_became_active":"2024-09-18T01:00:20.005875+0000","last_became_peered":"2024-09-18T01:00:20.005875+0000","last_unstale":"2024-09-18T01:00:30.208906+0000","last_undegraded":"2024-09-18T01:00:30.208906+0000","last_fullsized":"2024-09-18T01:00:30.208906+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:53:03.098519+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010647046,"stat_sum":{"num_bytes":8240308,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":26,"num_read":13301,"num_read_kb":40088,"num_write":925,"num_write_kb":11989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":22094834,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.1c","version":"172'601","reported_seq":4247,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.203791+0000","last_change":"2024-09-18T01:00:30.075791+0000","last_active":"2024-09-18T01:00:30.203791+0000","last_peered":"2024-09-18T01:00:30.203791+0000","last_clean":"2024-09-18T01:00:30.203791+0000","last_became_active":"2024-09-18T00:59:17.918223+0000","last_became_peered":"2024-09-18T00:59:17.918223+0000","last_unstale":"2024-09-18T01:00:30.203791+0000","last_undegraded":"2024-09-18T01:00:30.203791+0000","last_fullsized":"2024-09-18T01:00:30.203791+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:39:45.938031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083658880000000001,"stat_sum":{"num_bytes":2320457,"num_objects":33,"num_object_clones":4,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":19,"num_read":5409,"num_read_kb":27883,"num_write":672,"num_write_kb":11394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":5428683,"num_keys_recovered":92,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.1b","version":"172'1052","reported_seq":10402,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.221756+0000","last_change":"2024-09-18T01:00:30.076506+0000","last_active":"2024-09-18T01:00:30.221756+0000","last_peered":"2024-09-18T01:00:30.221756+0000","last_clean":"2024-09-18T01:00:30.221756+0000","last_became_active":"2024-09-18T01:00:22.941911+0000","last_became_peered":"2024-09-18T01:00:22.941911+0000","last_unstale":"2024-09-18T01:00:30.221756+0000","last_undegraded":"2024-09-18T01:00:30.221756+0000","last_fullsized":"2024-09-18T01:00:30.221756+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:20:20.642202+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.016477619999999998,"stat_sum":{"num_bytes":14660484,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":12624,"num_read_kb":40739,"num_write":1342,"num_write_kb":25586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":495,"num_bytes_recovered":59340116,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.1a","version":"172'331","reported_seq":4774,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.123587+0000","last_change":"2024-09-18T01:00:30.076845+0000","last_active":"2024-09-18T01:00:30.123587+0000","last_peered":"2024-09-18T01:00:30.123587+0000","last_clean":"2024-09-18T01:00:30.123587+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T01:00:30.123587+0000","last_undegraded":"2024-09-18T01:00:30.123587+0000","last_fullsized":"2024-09-18T01:00:30.123587+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:40.793569+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0094327360000000006,"stat_sum":{"num_bytes":25963863,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":17,"num_read":5062,"num_read_kb":29266,"num_write":297,"num_write_kb":12762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.19","version":"172'685","reported_seq":30720,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.208877+0000","last_change":"2024-09-18T01:00:30.075235+0000","last_active":"2024-09-18T01:00:30.208877+0000","last_peered":"2024-09-18T01:00:30.208877+0000","last_clean":"2024-09-18T01:00:30.208877+0000","last_became_active":"2024-09-18T01:00:10.952126+0000","last_became_peered":"2024-09-18T01:00:10.952126+0000","last_unstale":"2024-09-18T01:00:30.208877+0000","last_undegraded":"2024-09-18T01:00:30.208877+0000","last_fullsized":"2024-09-18T01:00:30.208877+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:48.373823+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0079566080000000004,"stat_sum":{"num_bytes":14584810,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":19,"num_read":30606,"num_read_kb":78256,"num_write":876,"num_write_kb":27079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":40291284,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.18","version":"172'612","reported_seq":3718,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.205175+0000","last_change":"2024-09-18T01:00:30.085585+0000","last_active":"2024-09-18T01:00:30.205175+0000","last_peered":"2024-09-18T01:00:30.205175+0000","last_clean":"2024-09-18T01:00:30.205175+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T01:00:30.205175+0000","last_undegraded":"2024-09-18T01:00:30.205175+0000","last_fullsized":"2024-09-18T01:00:30.205175+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:23:06.998046+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011587485,"stat_sum":{"num_bytes":28247344,"num_objects":30,"num_object_clones":2,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":20,"num_read":4562,"num_read_kb":13554,"num_write":841,"num_write_kb":42184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.17","version":"172'638","reported_seq":7100,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.212096+0000","last_change":"2024-09-18T01:00:30.079927+0000","last_active":"2024-09-18T01:00:30.212096+0000","last_peered":"2024-09-18T01:00:30.212096+0000","last_clean":"2024-09-18T01:00:30.212096+0000","last_became_active":"2024-09-18T01:00:20.017711+0000","last_became_peered":"2024-09-18T01:00:20.017711+0000","last_unstale":"2024-09-18T01:00:30.212096+0000","last_undegraded":"2024-09-18T01:00:30.212096+0000","last_fullsized":"2024-09-18T01:00:30.212096+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:25.051881+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093351719999999992,"stat_sum":{"num_bytes":7154449,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":14,"num_read":7116,"num_read_kb":46836,"num_write":750,"num_write_kb":25414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":13346337,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.16","version":"168'582","reported_seq":8406,"reported_epoch":171,"state":"peering","last_fresh":"2024-09-18T01:00:29.584058+0000","last_change":"2024-09-18T01:00:30.071402+0000","last_active":"2024-09-18T01:00:30.071402+0000","last_peered":"2024-09-18T01:00:29.584058+0000","last_clean":"2024-09-18T01:00:26.063502+0000","last_became_active":"2024-09-18T01:00:28.986307+0000","last_became_peered":"2024-09-18T01:00:28.986307+0000","last_unstale":"2024-09-18T01:00:29.584058+0000","last_undegraded":"2024-09-18T01:00:29.584058+0000","last_fullsized":"2024-09-18T01:00:28.937649+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.012553772,"stat_sum":{"num_bytes":7854719,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":8286,"num_read_kb":37063,"num_write":664,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":37601010,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"13","length":"6"}]},{"pgid":"3.15","version":"172'504","reported_seq":2882,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.070657+0000","last_change":"2024-09-18T01:00:30.070657+0000","last_active":"2024-09-18T01:00:30.070657+0000","last_peered":"2024-09-18T01:00:30.070657+0000","last_clean":"2024-09-18T01:00:30.070657+0000","last_became_active":"2024-09-18T01:00:21.007605+0000","last_became_peered":"2024-09-18T01:00:21.007605+0000","last_unstale":"2024-09-18T01:00:30.070657+0000","last_undegraded":"2024-09-18T01:00:30.070657+0000","last_fullsized":"2024-09-18T01:00:30.070657+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:03.056096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047018889999999999,"stat_sum":{"num_bytes":23482430,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":2322,"num_read_kb":15300,"num_write":800,"num_write_kb":48070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":22291504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.14","version":"168'366","reported_seq":2853,"reported_epoch":171,"state":"peering","last_fresh":"2024-09-18T01:00:29.134742+0000","last_change":"2024-09-18T01:00:30.060449+0000","last_active":"2024-09-18T01:00:30.060449+0000","last_peered":"2024-09-18T01:00:29.134742+0000","last_clean":"2024-09-18T01:00:26.066179+0000","last_became_active":"2024-09-18T01:00:28.986643+0000","last_became_peered":"2024-09-18T01:00:28.986643+0000","last_unstale":"2024-09-18T01:00:29.134742+0000","last_undegraded":"2024-09-18T01:00:29.134742+0000","last_fullsized":"2024-09-18T01:00:28.938461+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.0022479750000000001,"stat_sum":{"num_bytes":787963,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":16,"num_read":2449,"num_read_kb":3815,"num_write":338,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":469,"num_bytes_recovered":3474495,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"13","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":38,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.061712+0000","last_change":"2024-09-18T01:00:15.995790+0000","last_active":"2024-09-18T01:00:30.061712+0000","last_peered":"2024-09-18T01:00:30.061712+0000","last_clean":"2024-09-18T01:00:30.061712+0000","last_became_active":"2024-09-18T01:00:15.995425+0000","last_became_peered":"2024-09-18T01:00:15.995425+0000","last_unstale":"2024-09-18T01:00:30.061712+0000","last_undegraded":"2024-09-18T01:00:30.061712+0000","last_fullsized":"2024-09-18T01:00:30.061712+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:49.875513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'734","reported_seq":4687,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.213637+0000","last_change":"2024-09-18T01:00:30.083526+0000","last_active":"2024-09-18T01:00:30.213637+0000","last_peered":"2024-09-18T01:00:30.213637+0000","last_clean":"2024-09-18T01:00:30.213637+0000","last_became_active":"2024-09-18T01:00:11.936218+0000","last_became_peered":"2024-09-18T01:00:11.936218+0000","last_unstale":"2024-09-18T01:00:30.213637+0000","last_undegraded":"2024-09-18T01:00:30.213637+0000","last_fullsized":"2024-09-18T01:00:30.213637+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:57.644174+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.023677442,"stat_sum":{"num_bytes":10367537,"num_objects":35,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":21,"num_read":5455,"num_read_kb":22924,"num_write":980,"num_write_kb":4646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":2805113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":323,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.059645+0000","last_change":"2024-09-18T00:59:15.073941+0000","last_active":"2024-09-18T01:00:30.059645+0000","last_peered":"2024-09-18T01:00:30.059645+0000","last_clean":"2024-09-18T01:00:30.059645+0000","last_became_active":"2024-09-18T00:59:15.072986+0000","last_became_peered":"2024-09-18T00:59:15.072986+0000","last_unstale":"2024-09-18T01:00:30.059645+0000","last_undegraded":"2024-09-18T01:00:30.059645+0000","last_fullsized":"2024-09-18T01:00:30.059645+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:16.671308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"172'469","reported_seq":8628,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.211578+0000","last_change":"2024-09-18T01:00:30.079999+0000","last_active":"2024-09-18T01:00:30.211578+0000","last_peered":"2024-09-18T01:00:30.211578+0000","last_clean":"2024-09-18T01:00:30.211578+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T01:00:30.211578+0000","last_undegraded":"2024-09-18T01:00:30.211578+0000","last_fullsized":"2024-09-18T01:00:30.211578+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:42:11.845275+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.019932088000000001,"stat_sum":{"num_bytes":3511153,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":17,"num_read":8745,"num_read_kb":44785,"num_write":390,"num_write_kb":13858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":38,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.062721+0000","last_change":"2024-09-18T01:00:15.991240+0000","last_active":"2024-09-18T01:00:30.062721+0000","last_peered":"2024-09-18T01:00:30.062721+0000","last_clean":"2024-09-18T01:00:30.062721+0000","last_became_active":"2024-09-18T01:00:15.990930+0000","last_became_peered":"2024-09-18T01:00:15.990930+0000","last_unstale":"2024-09-18T01:00:30.062721+0000","last_undegraded":"2024-09-18T01:00:30.062721+0000","last_fullsized":"2024-09-18T01:00:30.062721+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:55.616429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"171'337","reported_seq":4332,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.065384+0000","last_change":"2024-09-18T01:00:30.068626+0000","last_active":"2024-09-18T01:00:30.065384+0000","last_peered":"2024-09-18T01:00:30.065384+0000","last_clean":"2024-09-18T01:00:30.065384+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T01:00:30.065384+0000","last_undegraded":"2024-09-18T01:00:30.065384+0000","last_fullsized":"2024-09-18T01:00:30.065384+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:24.368207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035266749999999999,"stat_sum":{"num_bytes":10021233,"num_objects":29,"num_object_clones":3,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":3644,"num_read_kb":27224,"num_write":284,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":317,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.073615+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T01:00:30.073615+0000","last_peered":"2024-09-18T01:00:30.073615+0000","last_clean":"2024-09-18T01:00:30.073615+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T01:00:30.073615+0000","last_undegraded":"2024-09-18T01:00:30.073615+0000","last_fullsized":"2024-09-18T01:00:30.073615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"172'545","reported_seq":4253,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.069511+0000","last_change":"2024-09-18T01:00:30.069511+0000","last_active":"2024-09-18T01:00:30.069511+0000","last_peered":"2024-09-18T01:00:30.069511+0000","last_clean":"2024-09-18T01:00:30.069511+0000","last_became_active":"2024-09-18T00:58:58.963883+0000","last_became_peered":"2024-09-18T00:58:58.963883+0000","last_unstale":"2024-09-18T01:00:30.069511+0000","last_undegraded":"2024-09-18T01:00:30.069511+0000","last_fullsized":"2024-09-18T01:00:30.069511+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:53:35.171244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00435404,"stat_sum":{"num_bytes":3586770,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":21,"num_read":4622,"num_read_kb":33385,"num_write":778,"num_write_kb":24919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.3","version":"172'866","reported_seq":5881,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.165676+0000","last_change":"2024-09-18T01:00:30.069986+0000","last_active":"2024-09-18T01:00:30.165676+0000","last_peered":"2024-09-18T01:00:30.165676+0000","last_clean":"2024-09-18T01:00:30.165676+0000","last_became_active":"2024-09-18T01:00:21.926403+0000","last_became_peered":"2024-09-18T01:00:21.926403+0000","last_unstale":"2024-09-18T01:00:30.165676+0000","last_undegraded":"2024-09-18T01:00:30.165676+0000","last_fullsized":"2024-09-18T01:00:30.165676+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:02:08.788378+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010226333000000001,"stat_sum":{"num_bytes":24798465,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":7535,"num_read_kb":19076,"num_write":1000,"num_write_kb":33900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":99850853,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"2.2","version":"23'2","reported_seq":176,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.064633+0000","last_change":"2024-09-18T01:00:28.086666+0000","last_active":"2024-09-18T01:00:30.064633+0000","last_peered":"2024-09-18T01:00:30.064633+0000","last_clean":"2024-09-18T01:00:30.064633+0000","last_became_active":"2024-09-18T01:00:28.078264+0000","last_became_peered":"2024-09-18T01:00:28.078264+0000","last_unstale":"2024-09-18T01:00:30.064633+0000","last_undegraded":"2024-09-18T01:00:30.064633+0000","last_fullsized":"2024-09-18T01:00:30.064633+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:15.282919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"172'611","reported_seq":8182,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.084437+0000","last_change":"2024-09-18T01:00:30.084791+0000","last_active":"2024-09-18T01:00:30.084437+0000","last_peered":"2024-09-18T01:00:30.084437+0000","last_clean":"2024-09-18T01:00:30.084437+0000","last_became_active":"2024-09-18T01:00:08.951992+0000","last_became_peered":"2024-09-18T01:00:08.951992+0000","last_unstale":"2024-09-18T01:00:30.084437+0000","last_undegraded":"2024-09-18T01:00:30.084437+0000","last_fullsized":"2024-09-18T01:00:30.084437+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:25:36.936651+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011109312,"stat_sum":{"num_bytes":9267931,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":26,"num_read":7620,"num_read_kb":29895,"num_write":722,"num_write_kb":19558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":15190015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":56,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-18T01:00:27.939016+0000","last_change":"2024-09-18T01:00:04.976725+0000","last_active":"2024-09-18T01:00:27.939016+0000","last_peered":"2024-09-18T01:00:27.939016+0000","last_clean":"2024-09-18T01:00:27.939016+0000","last_became_active":"2024-09-18T01:00:04.976360+0000","last_became_peered":"2024-09-18T01:00:04.976360+0000","last_unstale":"2024-09-18T01:00:27.939016+0000","last_undegraded":"2024-09-18T01:00:27.939016+0000","last_fullsized":"2024-09-18T01:00:27.939016+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:11.159126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"168'524","reported_seq":9230,"reported_epoch":171,"state":"peering","last_fresh":"2024-09-18T01:00:29.202819+0000","last_change":"2024-09-18T01:00:30.065974+0000","last_active":"2024-09-18T01:00:30.065974+0000","last_peered":"2024-09-18T01:00:29.202819+0000","last_clean":"2024-09-18T01:00:26.060099+0000","last_became_active":"2024-09-18T01:00:28.980566+0000","last_became_peered":"2024-09-18T01:00:28.980566+0000","last_unstale":"2024-09-18T01:00:29.202819+0000","last_undegraded":"2024-09-18T01:00:29.202819+0000","last_fullsized":"2024-09-18T01:00:28.934730+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.013998345000000001,"stat_sum":{"num_bytes":9486939,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":8919,"num_read_kb":48544,"num_write":547,"num_write_kb":18757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":794,"num_bytes_recovered":19034719,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"13","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":38,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.058052+0000","last_change":"2024-09-18T01:00:15.994517+0000","last_active":"2024-09-18T01:00:30.058052+0000","last_peered":"2024-09-18T01:00:30.058052+0000","last_clean":"2024-09-18T01:00:30.058052+0000","last_became_active":"2024-09-18T01:00:15.994159+0000","last_became_peered":"2024-09-18T01:00:15.994159+0000","last_unstale":"2024-09-18T01:00:30.058052+0000","last_undegraded":"2024-09-18T01:00:30.058052+0000","last_fullsized":"2024-09-18T01:00:30.058052+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:06.775713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"172'713","reported_seq":4935,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.062100+0000","last_change":"2024-09-18T01:00:30.062100+0000","last_active":"2024-09-18T01:00:30.062100+0000","last_peered":"2024-09-18T01:00:30.062100+0000","last_clean":"2024-09-18T01:00:30.062100+0000","last_became_active":"2024-09-18T00:59:18.918527+0000","last_became_peered":"2024-09-18T00:59:18.918527+0000","last_unstale":"2024-09-18T01:00:30.062100+0000","last_undegraded":"2024-09-18T01:00:30.062100+0000","last_fullsized":"2024-09-18T01:00:30.062100+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:44:14.216966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026017470000000002,"stat_sum":{"num_bytes":9585705,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":22,"num_read":5237,"num_read_kb":16438,"num_write":1137,"num_write_kb":52135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":12961092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"2.3","version":"21'1","reported_seq":51,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.058847+0000","last_change":"2024-09-18T01:00:28.080162+0000","last_active":"2024-09-18T01:00:30.058847+0000","last_peered":"2024-09-18T01:00:30.058847+0000","last_clean":"2024-09-18T01:00:30.058847+0000","last_became_active":"2024-09-18T01:00:28.075757+0000","last_became_peered":"2024-09-18T01:00:28.075757+0000","last_unstale":"2024-09-18T01:00:30.058847+0000","last_undegraded":"2024-09-18T01:00:30.058847+0000","last_fullsized":"2024-09-18T01:00:30.058847+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:29.327404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":469,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.066307+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T01:00:30.066307+0000","last_peered":"2024-09-18T01:00:30.066307+0000","last_clean":"2024-09-18T01:00:30.066307+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T01:00:30.066307+0000","last_undegraded":"2024-09-18T01:00:30.066307+0000","last_fullsized":"2024-09-18T01:00:30.066307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'433","reported_seq":5707,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.069398+0000","last_change":"2024-09-18T01:00:30.070060+0000","last_active":"2024-09-18T01:00:30.069398+0000","last_peered":"2024-09-18T01:00:30.069398+0000","last_clean":"2024-09-18T01:00:30.069398+0000","last_became_active":"2024-09-18T01:00:08.949384+0000","last_became_peered":"2024-09-18T01:00:08.949384+0000","last_unstale":"2024-09-18T01:00:30.069398+0000","last_undegraded":"2024-09-18T01:00:30.069398+0000","last_fullsized":"2024-09-18T01:00:30.069398+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:31.827882+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.010713089,"stat_sum":{"num_bytes":14680258,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":18,"num_read":5707,"num_read_kb":51783,"num_write":447,"num_write_kb":39546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":32435091,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.9","version":"168'394","reported_seq":3619,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.059316+0000","last_change":"2024-09-18T01:00:30.060635+0000","last_active":"2024-09-18T01:00:30.059316+0000","last_peered":"2024-09-18T01:00:30.059316+0000","last_clean":"2024-09-18T01:00:30.059316+0000","last_became_active":"2024-09-18T01:00:18.997710+0000","last_became_peered":"2024-09-18T01:00:18.997710+0000","last_unstale":"2024-09-18T01:00:30.059316+0000","last_undegraded":"2024-09-18T01:00:30.059316+0000","last_fullsized":"2024-09-18T01:00:30.059316+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:31.033378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021545750000000002,"stat_sum":{"num_bytes":9928288,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":21,"num_read":3625,"num_read_kb":11553,"num_write":338,"num_write_kb":23969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":27148173,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.a","version":"172'584","reported_seq":5457,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.218023+0000","last_change":"2024-09-18T01:00:30.069770+0000","last_active":"2024-09-18T01:00:30.218023+0000","last_peered":"2024-09-18T01:00:30.218023+0000","last_clean":"2024-09-18T01:00:30.218023+0000","last_became_active":"2024-09-18T01:00:22.938344+0000","last_became_peered":"2024-09-18T01:00:22.938344+0000","last_unstale":"2024-09-18T01:00:30.218023+0000","last_undegraded":"2024-09-18T01:00:30.218023+0000","last_fullsized":"2024-09-18T01:00:30.218023+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:21.349552+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010933154,"stat_sum":{"num_bytes":6755294,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":21,"num_read":4745,"num_read_kb":46810,"num_write":579,"num_write_kb":17196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":6639875,"num_keys_recovered":85,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.b","version":"168'457","reported_seq":5624,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-18T01:00:27.938914+0000","last_change":"2024-09-18T01:00:26.931492+0000","last_active":"2024-09-18T01:00:27.938914+0000","last_peered":"2024-09-18T01:00:27.938914+0000","last_clean":"2024-09-18T01:00:27.938914+0000","last_became_active":"2024-09-18T01:00:10.954910+0000","last_became_peered":"2024-09-18T01:00:10.954910+0000","last_unstale":"2024-09-18T01:00:27.938914+0000","last_undegraded":"2024-09-18T01:00:27.938914+0000","last_fullsized":"2024-09-18T01:00:27.938914+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:32:18.631544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070307200000000001,"stat_sum":{"num_bytes":11850172,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":21,"num_read":5292,"num_read_kb":66297,"num_write":467,"num_write_kb":15025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15399944,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.c","version":"170'529","reported_seq":6932,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-18T01:00:28.098707+0000","last_change":"2024-09-18T01:00:26.931729+0000","last_active":"2024-09-18T01:00:28.098707+0000","last_peered":"2024-09-18T01:00:28.098707+0000","last_clean":"2024-09-18T01:00:28.098707+0000","last_became_active":"2024-09-18T01:00:19.999750+0000","last_became_peered":"2024-09-18T01:00:19.999750+0000","last_unstale":"2024-09-18T01:00:28.098707+0000","last_undegraded":"2024-09-18T01:00:28.098707+0000","last_fullsized":"2024-09-18T01:00:28.098707+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:53:11.729276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086226399999999996,"stat_sum":{"num_bytes":5595948,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":6357,"num_read_kb":47360,"num_write":604,"num_write_kb":22372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":26783458,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"7"}]},{"pgid":"3.d","version":"172'914","reported_seq":6432,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.219951+0000","last_change":"2024-09-18T01:00:30.072741+0000","last_active":"2024-09-18T01:00:30.219951+0000","last_peered":"2024-09-18T01:00:30.219951+0000","last_clean":"2024-09-18T01:00:30.219951+0000","last_became_active":"2024-09-18T01:00:21.923279+0000","last_became_peered":"2024-09-18T01:00:21.923279+0000","last_unstale":"2024-09-18T01:00:30.219951+0000","last_undegraded":"2024-09-18T01:00:30.219951+0000","last_fullsized":"2024-09-18T01:00:30.219951+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":914,"log_dups_size":0,"ondisk_log_size":914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:29.431203+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010700220999999999,"stat_sum":{"num_bytes":4700247,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":20,"num_read":8220,"num_read_kb":40594,"num_write":1098,"num_write_kb":30729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":80305919,"num_keys_recovered":163,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.e","version":"172'683","reported_seq":4538,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.213642+0000","last_change":"2024-09-18T01:00:30.081644+0000","last_active":"2024-09-18T01:00:30.213642+0000","last_peered":"2024-09-18T01:00:30.213642+0000","last_clean":"2024-09-18T01:00:30.213642+0000","last_became_active":"2024-09-18T01:00:21.008903+0000","last_became_peered":"2024-09-18T01:00:21.008903+0000","last_unstale":"2024-09-18T01:00:30.213642+0000","last_undegraded":"2024-09-18T01:00:30.213642+0000","last_fullsized":"2024-09-18T01:00:30.213642+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:05:08.647258+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0091971659999999997,"stat_sum":{"num_bytes":680558,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":15,"num_read":4573,"num_read_kb":28954,"num_write":927,"num_write_kb":26460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":41587165,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.f","version":"171'537","reported_seq":4191,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.067161+0000","last_change":"2024-09-18T01:00:30.067804+0000","last_active":"2024-09-18T01:00:30.067161+0000","last_peered":"2024-09-18T01:00:30.067161+0000","last_clean":"2024-09-18T01:00:30.067161+0000","last_became_active":"2024-09-18T00:59:17.917905+0000","last_became_peered":"2024-09-18T00:59:17.917905+0000","last_unstale":"2024-09-18T01:00:30.067161+0000","last_undegraded":"2024-09-18T01:00:30.067161+0000","last_fullsized":"2024-09-18T01:00:30.067161+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:59:31.730214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001161498,"stat_sum":{"num_bytes":9001370,"num_objects":25,"num_object_clones":1,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":5267,"num_read_kb":28161,"num_write":572,"num_write_kb":9282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":6320640,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.10","version":"172'408","reported_seq":5005,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.209400+0000","last_change":"2024-09-18T01:00:30.068464+0000","last_active":"2024-09-18T01:00:30.209400+0000","last_peered":"2024-09-18T01:00:30.209400+0000","last_clean":"2024-09-18T01:00:30.209400+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T01:00:30.209400+0000","last_undegraded":"2024-09-18T01:00:30.209400+0000","last_fullsized":"2024-09-18T01:00:30.209400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:12:18.835805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022946440000000002,"stat_sum":{"num_bytes":30277407,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":4913,"num_read_kb":51340,"num_write":328,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.11","version":"172'281","reported_seq":3085,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.063324+0000","last_change":"2024-09-18T01:00:30.063324+0000","last_active":"2024-09-18T01:00:30.063324+0000","last_peered":"2024-09-18T01:00:30.063324+0000","last_clean":"2024-09-18T01:00:30.063324+0000","last_became_active":"2024-09-18T01:00:19.002365+0000","last_became_peered":"2024-09-18T01:00:19.002365+0000","last_unstale":"2024-09-18T01:00:30.063324+0000","last_undegraded":"2024-09-18T01:00:30.063324+0000","last_fullsized":"2024-09-18T01:00:30.063324+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:26:05.692966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041361979999999998,"stat_sum":{"num_bytes":12846580,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":12,"num_read":1897,"num_read_kb":13175,"num_write":239,"num_write_kb":14260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":15824068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.12","version":"172'611","reported_seq":6732,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.208542+0000","last_change":"2024-09-18T01:00:30.075294+0000","last_active":"2024-09-18T01:00:30.208542+0000","last_peered":"2024-09-18T01:00:30.208542+0000","last_clean":"2024-09-18T01:00:30.208542+0000","last_became_active":"2024-09-18T01:00:21.926814+0000","last_became_peered":"2024-09-18T01:00:21.926814+0000","last_unstale":"2024-09-18T01:00:30.208542+0000","last_undegraded":"2024-09-18T01:00:30.208542+0000","last_fullsized":"2024-09-18T01:00:30.208542+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:32:50.112646+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.010872308000000001,"stat_sum":{"num_bytes":10866641,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":18,"num_read":6923,"num_read_kb":17106,"num_write":735,"num_write_kb":11280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":32804395,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.13","version":"172'473","reported_seq":4336,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-18T01:00:30.064868+0000","last_change":"2024-09-18T01:00:30.064868+0000","last_active":"2024-09-18T01:00:30.064868+0000","last_peered":"2024-09-18T01:00:30.064868+0000","last_clean":"2024-09-18T01:00:30.064868+0000","last_became_active":"2024-09-18T01:00:23.943957+0000","last_became_peered":"2024-09-18T01:00:23.943957+0000","last_unstale":"2024-09-18T01:00:30.064868+0000","last_undegraded":"2024-09-18T01:00:30.064868+0000","last_fullsized":"2024-09-18T01:00:30.064868+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:51:35.473324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056838849999999996,"stat_sum":{"num_bytes":1572910,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":4856,"num_read_kb":26580,"num_write":551,"num_write_kb":11729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":9379374,"num_keys_recovered":69,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":339188792,"num_objects":1008,"num_object_clones":140,"num_object_copies":2016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1008,"num_whiteouts":602,"num_read":217142,"num_read_kb":1064338,"num_write":21535,"num_write_kb":688405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9848,"num_bytes_recovered":693846180,"num_keys_recovered":1158,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":423563264,"data_stored":485420700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18483,"ondisk_log_size":18483,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72496,"kb_used_data":27420,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94299344,"statfs":{"total":96636764160,"available":96562528256,"internally_reserved":0,"allocated":28078080,"data_stored":27152762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87388,"kb_used_data":42308,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94284452,"statfs":{"total":96636764160,"available":96547278848,"internally_reserved":0,"allocated":43323392,"data_stored":45373065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103104,"kb_used_data":49700,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94268736,"statfs":{"total":96636764160,"available":96531185664,"internally_reserved":0,"allocated":50892800,"data_stored":52975615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107396,"kb_used_data":58124,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94264444,"statfs":{"total":96636764160,"available":96526790656,"internally_reserved":0,"allocated":59518976,"data_stored":61896753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":64424509483,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89832,"kb_used_data":40560,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94282008,"statfs":{"total":96636764160,"available":96544776192,"internally_reserved":0,"allocated":41533440,"data_stored":41905266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":64424509482,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68816,"kb_used_data":23732,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94303024,"statfs":{"total":96636764160,"available":96566296576,"internally_reserved":0,"allocated":24301568,"data_stored":24870529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136798208,"data_stored":170151383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38871040,"data_stored":40325250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52674560,"data_stored":68201235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57454592,"data_stored":60907124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":48828416,"data_stored":51985986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40665088,"data_stored":43793068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":26009600,"data_stored":26163114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22261760,"data_stored":23893540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:00:33.620 INFO:tasks.thrashosds.thrasher:Setting 3.0 to [3, 5, 4, 1] 2024-09-18T01:00:33.620 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.0', '3', '5', '4', '1'] 2024-09-18T01:00:33.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.0 3 5 4 1 2024-09-18T01:00:33.622 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:33.622 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:33.638 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:33.639 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:33.655 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:33.655 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:33.665 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:33.665 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:33.676 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:33.676 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:33.679 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:00:33.697 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:33.697 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:33.699 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:33.696+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:33.717 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:33.717 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:33.738 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:33.739 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:33.799 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:33.800+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:33.811 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:33.812 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:33.826 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:00:33.900 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:33.896+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:33.937 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:33.937 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:33.949 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:00:34.038 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:34.036+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:34.074 INFO:teuthology.orchestra.run.smithi042.stderr:set 3.0 pg_upmap_items mapping to [3->5,4->1] 2024-09-18T01:00:34.103 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:34.100+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:34.125 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:34.138 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:00:34.204 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:34.200+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:34.281 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:34.281 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:34.296 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:00:34.344 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:34.344+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:34.407 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:34.404+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:34.413 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:34.414 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:34.428 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:00:34.508 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:34.504+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:34.600 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:34.600 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:34.609 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:34.604+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:34.614 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:00:34.710 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:34.708+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:34.732 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:34.732 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:34.749 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:00:34.811 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:34.808+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:34.866 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:34.866 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:34.880 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:00:34.924 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:34.924+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:35.014 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:35.012+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:35.049 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:35.049 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:35.064 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:00:35.115 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:35.112+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:35.184 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:00:35.196 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:35.196 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:35.213 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:00:35.256 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:35.256+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:35.317 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:35.312+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:35.374 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:35.375 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:35.389 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:00:35.418 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:35.416+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:35.509 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:35.509 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:35.519 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:35.516+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:35.523 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:00:35.620 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:35.620+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:35.690 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:35.690 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:35.704 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:00:35.748 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:35.748+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:35.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:00:35.821 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:35.816+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:35.867 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:35.867 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:35.882 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:00:35.922 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:35.920+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:35.923 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:35.923 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:35.940 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:35.957 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:35.974 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:35.974 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:35.994 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:36.008 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:36.008 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:36.014 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:36.015 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:36.023 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:36.020+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:36.023 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:00:36.033 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:36.033 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:36.052 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:36.052 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:36.088 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-18T01:00:36.158 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:36.159 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:36.159 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:36.159 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:36.159 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:36.156+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:36.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:00:36.266 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:36.264+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:36.324 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:36.324+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:36.351 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:36.351 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:36.351 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:36.351 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:36.351 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:36.366 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:00:36.426 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:36.424+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:36.527 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:36.524+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:36.540 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 167.473138081, 2024-09-18T01:00:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T01:00:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T01:00:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T01:00:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.580604261999994, 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:00:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T01:00:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.831537471999994, 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.821163016, 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T01:00:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T01:00:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.821090334000004, 2024-09-18T01:00:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T01:00:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T01:00:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T01:00:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.821059468000001, 2024-09-18T01:00:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T01:00:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T01:00:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.821006694000005, 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T01:00:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.820980528000007, 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T01:00:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.820955373000004, 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T01:00:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.820532061999998, 2024-09-18T01:00:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T01:00:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T01:00:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.820493683999999, 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T01:00:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T01:00:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T01:00:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T01:00:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.820468919000007, 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T01:00:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T01:00:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T01:00:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T01:00:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T01:00:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T01:00:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T01:00:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T01:00:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T01:00:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T01:00:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T01:00:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T01:00:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.813674155000001, 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.813648555, 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.813625212000005, 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T01:00:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.813602773, 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.813581999999997, 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T01:00:36.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T01:00:36.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T01:00:36.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T01:00:36.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.623 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.813560973999998, 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.624 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T01:00:36.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.813520392000001, 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T01:00:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T01:00:36.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.813486292999997, 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:36.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T01:00:36.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.637 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.637 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.637 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:36.637 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:36.637 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:36.638 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:00:36.639 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:36.636+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:36.739 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:36.736+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:36.814 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:36.814 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:36.814 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:36.828 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:00:36.840 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:36.840+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:36.941 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:36.936+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:36.997 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:36.997 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:36.997 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:36.997 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:36.997 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:37.012 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:00:37.042 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:37.040+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:37.143 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:37.140+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 193.90300775399999, 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T01:00:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T01:00:37.199 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T01:00:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:37.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T01:00:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T01:00:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T01:00:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T01:00:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T01:00:37.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T01:00:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T01:00:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T01:00:37.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T01:00:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T01:00:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T01:00:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T01:00:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T01:00:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T01:00:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T01:00:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T01:00:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T01:00:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T01:00:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T01:00:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T01:00:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T01:00:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T01:00:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T01:00:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T01:00:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T01:00:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T01:00:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.212 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.706645264, 2024-09-18T01:00:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T01:00:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.213 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T01:00:37.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T01:00:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T01:00:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T01:00:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T01:00:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T01:00:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T01:00:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T01:00:37.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T01:00:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T01:00:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T01:00:37.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T01:00:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T01:00:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T01:00:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T01:00:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.70654993700001, 2024-09-18T01:00:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T01:00:37.223 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.224 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T01:00:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:37.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T01:00:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T01:00:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T01:00:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T01:00:37.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T01:00:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T01:00:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T01:00:37.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T01:00:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T01:00:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T01:00:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T01:00:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T01:00:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T01:00:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.232 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.70652449799999, 2024-09-18T01:00:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T01:00:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.233 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T01:00:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.234 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:37.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T01:00:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T01:00:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T01:00:37.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T01:00:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T01:00:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T01:00:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T01:00:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T01:00:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T01:00:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T01:00:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T01:00:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T01:00:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T01:00:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T01:00:37.241 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.242 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.70650197500001, 2024-09-18T01:00:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T01:00:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T01:00:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:00:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T01:00:37.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.706478535, 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T01:00:37.249 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T01:00:37.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T01:00:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T01:00:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T01:00:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T01:00:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T01:00:37.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.70645472499999, 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.258 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.70643236000001, 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.706405743, 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.70218766400001, 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T01:00:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T01:00:37.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.70216132300001, 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.277 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.70213865299999, 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T01:00:37.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T01:00:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T01:00:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T01:00:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T01:00:37.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T01:00:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T01:00:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T01:00:37.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.70212028, 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.283 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T01:00:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.287 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.702100492, 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T01:00:37.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:37.292 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 168.702073622, 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T01:00:37.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 121.625883153, 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.297 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T01:00:37.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T01:00:37.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 80.214818057000002, 2024-09-18T01:00:37.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T01:00:37.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T01:00:37.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:140635 3.d 3:b08054e4:::rbd_object_map.117ced55205b.0000000000000023:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:37.310 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.853704663000002, 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86222047999999996, 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 140635 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338625+0000", 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6199999999999999e-06 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338634+0000", 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9099999999999994e-06 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338657+0000", 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3672999999999998e-05 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338675+0000", 2024-09-18T01:00:37.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7589999999999999e-05 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338767+0000", 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1896000000000002e-05 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338776+0000", 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.38e-06 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200213+0000", 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86143668200000001 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200432+0000", 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021884199999999999 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200844+0000", 2024-09-18T01:00:37.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000411887 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:155664 3.d 3:b496760f:::rbd_data.117c1d78ece1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.286401558999998, 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046053647000000003, 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:37.316 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 155664 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:37.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905927+0000", 2024-09-18T01:00:37.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.175e-06 2024-09-18T01:00:37.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909267+0000", 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003339266 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909287+0000", 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605e-05 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909297+0000", 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3139999999999993e-06 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909363+0000", 2024-09-18T01:00:37.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6437000000000001e-05 2024-09-18T01:00:37.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909483+0000", 2024-09-18T01:00:37.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119861 2024-09-18T01:00:37.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909868+0000", 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038564099999999999 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.917828+0000", 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079592570000000008 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951788+0000", 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033960314999999998 2024-09-18T01:00:37.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951895+0000", 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107316 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951980+0000", 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4460000000000001e-05 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.322 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:161321 3.d 3:b70a3213:::rbd_header.117c59271281:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e151)", 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.08876266, 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.84105173099999997, 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 161321 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103567+0000", 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.553e-06 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103576+0000", 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4099999999999997e-06 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103601+0000", 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4833000000000001e-05 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103620+0000", 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9409999999999999e-05 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103789+0000", 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016871599999999999 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103807+0000", 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8502999999999999e-05 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.943761+0000", 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.83995325200000004 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.944023+0000", 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026232799999999999 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.944617+0000", 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059372600000000004 2024-09-18T01:00:37.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.328 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.328 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.328 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.328 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.328 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:37.329 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:00:37.330 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:37.324+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:37.430 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:37.428+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:37.505 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:37.505 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:37.506 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:37.519 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:00:37.562 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:37.560+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:37.632 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:37.628+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:37.683 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:37.683 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:37.683 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:37.698 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:00:37.742 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:37.740+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:37.833 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:37.832+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:37.874 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:37.874 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:37.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:37.874 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:37.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.793393251, 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T01:00:37.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T01:00:37.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.79246366000001, 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.879 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T01:00:37.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T01:00:37.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.883 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.7924496, 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.887 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.79115902500001, 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T01:00:37.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T01:00:37.892 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.791138553, 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T01:00:37.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.896 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.78834409999999, 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:37.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:37.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:00:37.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T01:00:37.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T01:00:37.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.78832891600001, 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.902 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T01:00:37.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.903 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T01:00:37.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T01:00:37.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T01:00:37.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.78830597000001, 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T01:00:37.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.911 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.78412833100001, 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T01:00:37.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T01:00:37.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.78410711399999, 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.916 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.920 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.784085731, 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T01:00:37.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.78018587700001, 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T01:00:37.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.78017167600001, 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T01:00:37.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T01:00:37.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.78015363899999, 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T01:00:37.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.78013957300001, 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T01:00:37.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T01:00:37.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.77611422199999, 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.943 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:37.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.77610021000001, 2024-09-18T01:00:37.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:37.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T01:00:37.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T01:00:37.951 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.77608624600001, 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.952 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T01:00:37.956 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 169.01422829200001, 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T01:00:37.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.88875192099999, 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:37.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:37.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T01:00:37.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:37.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:37.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:37.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:37.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T01:00:37.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T01:00:37.967 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.967 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.967 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.967 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:37.967 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:37.967 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:37.969 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:00:37.969 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:37.964+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:38.070 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:38.068+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:38.071 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:00:38.115 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:00:38.160 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:38.161 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:38.171 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:38.168+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:38.179 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:00:38.236 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:38.236 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:38.258 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:38.258 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:38.271 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:38.268+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:38.289 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:38.289 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:38.306 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:38.306 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:38.327 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:38.327 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:38.348 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:38.348 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:38.350 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:38.350 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:38.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:00:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:38.370 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:38.390 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:38.390 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:38.391 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:38.388+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:38.473 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:38.468+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:38.484 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:38.484 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:38.498 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:00:38.575 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:38.572+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:38.633 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:38.633 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:38.647 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:00:38.692 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:38.692+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:38.770 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:38.770 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:38.777 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:38.776+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:38.784 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:00:38.881 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:38.876+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:38.951 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:38.951 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:38.965 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:00:39.008 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:39.008+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:39.080 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:39.080+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:39.103 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6, 7, 3] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:00:39.103 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:00:39.103 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-18T01:00:39.104 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:00:39.104 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:00:39.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-18T01:00:39.104 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-18T01:00:39.137 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:39.137 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:39.152 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:00:39.182 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:39.180+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:39.244 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:39.244 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T01:00:39.244 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:39.302 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:39.300+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:39.326 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:39.327 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:39.341 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:00:39.382 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:39.380+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:39.462 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:39.462 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:39.476 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:00:39.483 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:39.480+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:39.583 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:39.580+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:39.598 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:39.599 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:39.613 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:00:39.685 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:39.680+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:39.736 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:39.736 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:39.751 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:00:39.799 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:39.800+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:39.871 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:39.871 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:39.886 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:00:39.886 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:39.884+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:39.987 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:39.984+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:40.047 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:40.047 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:40.061 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:00:40.112 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:40.112+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:40.188 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:40.184+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:40.198 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:40.198 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:40.215 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:00:40.289 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:40.284+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:40.388 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:40.388 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:40.390 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:40.388+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:40.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:00:40.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:00:40.491 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:40.488+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:40.540 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:40.540 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:40.556 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:00:40.591 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:40.592+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:40.595 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:40.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:40.612 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:40.629 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:40.647 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:40.647 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:40.668 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:40.668 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:40.674 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:40.674 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:40.674 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:40.674 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:40.674 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:40.688 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:00:40.689 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:40.689 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:40.691 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:40.692+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:40.711 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:40.711 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:40.733 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:40.734 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:40.798 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:40.796+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:40.810 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:40.810 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:40.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:40.810 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:40.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.810 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:9807 3.15 3:ac8753f7:::rbd_data.117c918d6242.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 26214400~33453] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:40.810 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:40.810 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 171.742409345, 2024-09-18T01:00:40.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.14010198600000001, 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 9807 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063965+0000", 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.063969+0000", 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7000000000000002e-06 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082683+0000", 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.018714214999999999 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082721+0000", 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8185000000000001e-05 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082734+0000", 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2593e-05 2024-09-18T01:00:40.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.082867+0000", 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013322900000000001 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.083131+0000", 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026400600000000001 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 6", 2024-09-18T01:00:40.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.086559+0000", 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0034277309999999998 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.157630+0000", 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.071071016000000001 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203739+0000", 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046109606999999997 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:40.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.203929+0000", 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000189755 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:49.204067+0000", 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137949 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98703 3.e 3:75394fc3:::rbd_object_map.117c8e0d79e4:head [read 0~139] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:40.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.84987552600001, 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0110038560000001, 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98703 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956499+0000", 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956500+0000", 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956506+0000", 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2900000000000002e-06 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956521+0000", 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5288e-05 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.956529+0000", 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958147+0000", 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001618237 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:57.958166+0000", 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8468999999999999e-05 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.966748+0000", 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.008582426 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967071+0000", 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032302200000000001 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:58.967503+0000", 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00043129300000000002 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:125750 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.100808736000005, 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078137439000000003, 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 125750 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705566+0000", 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8500000000000002e-07 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705568+0000", 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4059999999999999e-06 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705575+0000", 2024-09-18T01:00:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6420000000000001e-06 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.705581+0000", 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.49e-06 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783518+0000", 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077937638000000004 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783593+0000", 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4738000000000002e-05 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.783703+0000", 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011014 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126227 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.090434279999997, 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078154322999999998, 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126227 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715940+0000", 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715941+0000", 2024-09-18T01:00:40.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715943+0000", 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6460000000000002e-06 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715950+0000", 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9990000000000002e-06 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.715956+0000", 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4949999999999999e-06 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.793947+0000", 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077990872000000003 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794019+0000", 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2208999999999999e-05 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794094+0000", 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000004e-05 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126229 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.090361598000001, 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078224076000000003, 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126229 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716013+0000", 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716014+0000", 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716017+0000", 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9450000000000003e-06 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716027+0000", 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1710000000000008e-06 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716033+0000", 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5690000000000001e-06 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794125+0000", 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078092010000000003 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794171+0000", 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5497000000000002e-05 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794237+0000", 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6129000000000003e-05 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.835 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126231 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.090330731999998, 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078415851999999994, 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126231 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716044+0000", 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716071+0000", 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7027999999999999e-05 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716080+0000", 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7530000000000007e-06 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716086+0000", 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794282+0000", 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078195925999999999 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794375+0000", 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3078999999999994e-05 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794459+0000", 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4054000000000005e-05 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126233 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.090277958000001, 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078562208999999994, 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126233 2024-09-18T01:00:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:40.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716096+0000", 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716097+0000", 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.37e-07 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716100+0000", 2024-09-18T01:00:40.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8430000000000001e-06 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716107+0000", 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1189999999999999e-06 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716114+0000", 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9330000000000004e-06 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794558+0000", 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078444046000000003 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794615+0000", 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6554000000000001e-05 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794659+0000", 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3977000000000003e-05 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126235 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.090251792000004, 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078659801000000001, 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126235 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716123+0000", 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716126+0000", 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.576e-06 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716134+0000", 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6449999999999994e-06 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716141+0000", 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2689999999999996e-06 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794689+0000", 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548435 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794729+0000", 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9867999999999998e-05 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794782+0000", 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3548999999999998e-05 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126237 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.090226637000001, 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078792434999999994, 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126237 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716148+0000", 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716151+0000", 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5210000000000001e-06 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716157+0000", 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7850000000000003e-06 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716163+0000", 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4609999999999999e-06 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794812+0000", 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078649294999999994 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794869+0000", 2024-09-18T01:00:40.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6434999999999998e-05 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794940+0000", 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126239 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:40.853 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.089803325999995, 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078548670000000001, 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126239 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716571+0000", 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716572+0000", 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5600000000000005e-07 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716576+0000", 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000003e-06 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716590+0000", 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3427999999999999e-05 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716597+0000", 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.374e-06 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.794978+0000", 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078380790000000006 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795043+0000", 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5086999999999995e-05 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795120+0000", 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6575000000000005e-05 2024-09-18T01:00:40.857 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126241 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.089764947999996, 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078695719999999997, 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126241 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716609+0000", 2024-09-18T01:00:40.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716610+0000", 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716613+0000", 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716621+0000", 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6299999999999998e-06 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716627+0000", 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8760000000000002e-06 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795152+0000", 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078525867999999999 2024-09-18T01:00:40.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795233+0000", 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0129999999999993e-05 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795305+0000", 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2591999999999999e-05 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126243 3.13 3:cfcf19c2:::rbd_data.117c8e0d79e4.0000000000000018:20 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.089740183000004, 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078886041000000004, 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126243 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716634+0000", 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716635+0000", 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1199999999999998e-07 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716637+0000", 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5959999999999998e-06 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716644+0000", 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6220000000000003e-06 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.716650+0000", 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000001e-06 2024-09-18T01:00:40.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795373+0000", 2024-09-18T01:00:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078723839000000004 2024-09-18T01:00:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795452+0000", 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8700000000000002e-05 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.795520+0000", 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8132000000000004e-05 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126419 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.082945418999998, 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078093768999999993, 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126419 2024-09-18T01:00:40.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723429+0000", 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3000000000000001e-07 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723432+0000", 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.988e-06 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723441+0000", 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0700000000000007e-06 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723447+0000", 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.071e-06 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801123+0000", 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.077676077999999996 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801198+0000", 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5313000000000002e-05 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801523+0000", 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032471900000000001 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126420 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.082919818999997, 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078371679999999999, 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126420 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723455+0000", 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.89e-07 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723458+0000", 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4779999999999998e-06 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723464+0000", 2024-09-18T01:00:40.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723471+0000", 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4849999999999997e-06 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801568+0000", 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078097258000000003 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801653+0000", 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4645e-05 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801826+0000", 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017372199999999999 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126421 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.082896476000002, 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078628172999999996, 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126421 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723478+0000", 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723479+0000", 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723481+0000", 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3769999999999999e-06 2024-09-18T01:00:40.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723488+0000", 2024-09-18T01:00:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7050000000000002e-06 2024-09-18T01:00:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723493+0000", 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801862+0000", 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078368425000000005 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.801940+0000", 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8226000000000006e-05 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802106+0000", 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016613899999999999 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126422 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.082874036999996, 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078909541999999999, 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126422 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723500+0000", 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723501+0000", 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723503+0000", 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4279999999999999e-06 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723510+0000", 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723515+0000", 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9609999999999996e-06 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802168+0000", 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078653371999999999 2024-09-18T01:00:40.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802246+0000", 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8337000000000006e-05 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802410+0000", 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016341500000000001 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126423 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.082853263999993, 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079190003999999994, 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126423 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.887 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723521+0000", 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723522+0000", 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.63e-07 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723524+0000", 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723530+0000", 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3489999999999997e-06 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723535+0000", 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802456+0000", 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.078920877 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802534+0000", 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8256000000000004e-05 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802711+0000", 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000177119 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126424 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.082832237999995, 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079443801999999994, 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126424 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723542+0000", 2024-09-18T01:00:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723543+0000", 2024-09-18T01:00:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:00:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723545+0000", 2024-09-18T01:00:40.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7149999999999998e-06 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723565+0000", 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9304999999999999e-05 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723573+0000", 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3809999999999999e-06 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802744+0000", 2024-09-18T01:00:40.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079170632000000005 2024-09-18T01:00:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802814+0000", 2024-09-18T01:00:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0294000000000002e-05 2024-09-18T01:00:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.802986+0000", 2024-09-18T01:00:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000172032 2024-09-18T01:00:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126425 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 487424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.082791655999998, 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080139538999999996, 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126425 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723583+0000", 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723593+0000", 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7799999999999995e-06 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723601+0000", 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8269999999999998e-06 2024-09-18T01:00:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723608+0000", 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803024+0000", 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.079415642999999994 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803099+0000", 2024-09-18T01:00:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5247000000000006e-05 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803722+0000", 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062294 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:126426 3.13 3:cd23ee3b:::rbd_data.117c8e0d79e4.000000000000000c:20 [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.082757556999994, 2024-09-18T01:00:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080560149999999997, 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 126426 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723617+0000", 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723620+0000", 2024-09-18T01:00:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4669999999999998e-06 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723627+0000", 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6810000000000001e-06 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.723632+0000", 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3299999999999998e-06 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803912+0000", 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080280529000000003 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.803998+0000", 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5779999999999995e-05 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:08.804177+0000", 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882900000000001 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:40.974 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:40.975 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:00:40.976 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:40.972+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:41.076 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:41.072+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:41.135 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:00:41.158 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:41.158 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:41.158 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:41.159 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:41.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:00:41.222 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:41.220+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:41.279 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:41.276+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:41.360 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:41.361 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:41.361 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:41.361 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:41.361 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:41.376 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:00:41.379 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-18T01:00:41.376+0000 7f833975e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26593) UID: 0 2024-09-18T01:00:41.481 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:41.476+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:41.495 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 198.20435965499999, 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.19019798900000001, 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.496 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4204", 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.195:0/2512818645", 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289320+0000", 2024-09-18T01:00:41.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289323+0000", 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.74e-06 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289336+0000", 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.322e-05 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289359+0000", 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2716000000000001e-05 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289377+0000", 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289570+0000", 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019370300000000001 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:41.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.289604+0000", 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3702999999999999e-05 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.476735+0000", 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.18713153599999999 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477117+0000", 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038181000000000001 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.477431+0000", 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00031408800000000002 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.478644+0000", 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001212344 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479348+0000", 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070388799999999995 2024-09-18T01:00:41.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479449+0000", 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101822 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:23.479518+0000", 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8540000000000004e-05 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7438 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.00799716500001, 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045702482000000003, 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7438 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485682+0000", 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485683+0000", 2024-09-18T01:00:41.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.15e-07 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485685+0000", 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3209999999999999e-06 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485758+0000", 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3287999999999994e-05 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485766+0000", 2024-09-18T01:00:41.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0940000000000002e-06 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531235+0000", 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045469016000000001 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531315+0000", 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0648000000000004e-05 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531385+0000", 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.97e-05 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7439 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1048576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.00790183800001, 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045790668, 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7439 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485778+0000", 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7199999999999998e-07 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485782+0000", 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6619999999999998e-06 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485790+0000", 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0490000000000004e-06 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485796+0000", 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6749999999999999e-06 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531418+0000", 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045621706999999997 2024-09-18T01:00:41.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531498+0000", 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0050999999999996e-05 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531568+0000", 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0852000000000005e-05 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.512 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7440 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.512 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:41.512 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.007876399, 2024-09-18T01:00:41.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045930957000000001, 2024-09-18T01:00:41.512 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.512 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.512 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.513 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7440 2024-09-18T01:00:41.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.513 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:41.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485803+0000", 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485804+0000", 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485806+0000", 2024-09-18T01:00:41.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2340000000000001e-06 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485813+0000", 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0890000000000002e-06 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485819+0000", 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.75e-06 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531599+0000", 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045779891000000003 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531672+0000", 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2891000000000004e-05 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531734+0000", 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2645999999999996e-05 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7441 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1179648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.00785387600001, 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046087597000000001, 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.517 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7441 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:41.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485826+0000", 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485829+0000", 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0450000000000001e-06 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485836+0000", 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8460000000000002e-06 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485842+0000", 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4970000000000001e-06 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531762+0000", 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045920713000000002 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531841+0000", 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8453000000000003e-05 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531913+0000", 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2605e-05 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.521 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7452 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.00783043600001, 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046363346999999999, 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7452 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485849+0000", 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485850+0000", 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485852+0000", 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.26e-06 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485859+0000", 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485864+0000", 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-18T01:00:41.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.531942+0000", 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046077931000000003 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532094+0000", 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015153399999999999 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532213+0000", 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000118647 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7456 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 65536~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.00780662599999, 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046543609999999999, 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7456 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485873+0000", 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485874+0000", 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485876+0000", 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485883+0000", 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485888+0000", 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2220000000000001e-06 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532249+0000", 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046361566 2024-09-18T01:00:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532328+0000", 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8819000000000005e-05 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532417+0000", 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8448000000000004e-05 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7458 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 131072~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.00778426100001, 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046710651999999998, 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7458 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485895+0000", 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485896+0000", 2024-09-18T01:00:41.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485898+0000", 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1229999999999998e-06 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485910+0000", 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2035000000000001e-05 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485915+0000", 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532445+0000", 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046529918000000003 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532517+0000", 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1605000000000003e-05 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532606+0000", 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9245000000000004e-05 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7459 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1212416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:41.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.00775764400001, 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046852998, 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7459 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.485922+0000", 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489747+0000", 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038248819999999999 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489757+0000", 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0479000000000001e-05 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489772+0000", 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4218e-05 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.489782+0000", 2024-09-18T01:00:41.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0159e-05 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532633+0000", 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.042850913999999997 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532705+0000", 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2280999999999995e-05 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.532775+0000", 2024-09-18T01:00:41.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0065000000000001e-05 2024-09-18T01:00:41.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7514 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1835008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.540 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:41.540 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.00353956500001, 2024-09-18T01:00:41.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045727565999999997, 2024-09-18T01:00:41.540 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7514 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490140+0000", 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490141+0000", 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1999999999999998e-07 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490144+0000", 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8650000000000001e-06 2024-09-18T01:00:41.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490152+0000", 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8469999999999996e-06 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490158+0000", 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3199999999999996e-06 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535729+0000", 2024-09-18T01:00:41.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045571109999999998 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535799+0000", 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9577000000000001e-05 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535868+0000", 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9027000000000002e-05 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7515 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.00351322399999, 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045870155000000003, 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7515 2024-09-18T01:00:41.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490166+0000", 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490167+0000", 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7100000000000002e-07 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490170+0000", 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249999999999998e-06 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490177+0000", 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5320000000000003e-06 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490182+0000", 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0189999999999997e-06 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535896+0000", 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045714101 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.535972+0000", 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6811999999999996e-05 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536037+0000", 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4095000000000002e-05 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7516 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1966080~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.003490554, 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046023461000000002, 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7516 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490189+0000", 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8999999999999998e-07 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490191+0000", 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.717e-06 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490198+0000", 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5980000000000002e-06 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490203+0000", 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5099999999999998e-06 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536068+0000", 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045864522999999997 2024-09-18T01:00:41.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536137+0000", 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9031999999999999e-05 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536213+0000", 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5791000000000007e-05 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7517 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 2031616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.00347218100001, 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046171834000000002, 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7517 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490207+0000", 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490208+0000", 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:00:41.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490209+0000", 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.266e-06 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490214+0000", 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3600000000000004e-06 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490219+0000", 2024-09-18T01:00:41.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.651e-06 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536240+0000", 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046020739999999997 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536310+0000", 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0259999999999995e-05 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536379+0000", 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9311000000000001e-05 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7532 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.003452393, 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046343269999999999, 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7532 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490227+0000", 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490228+0000", 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490236+0000", 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490244+0000", 2024-09-18T01:00:41.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4240000000000004e-06 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490247+0000", 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.585e-06 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536406+0000", 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046159172999999998 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536478+0000", 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1816999999999994e-05 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536570+0000", 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2361999999999994e-05 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.562 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:7534 3.10 3:0df3ad4b:::rbd_data.117ce245106a.0000000000000011:5 [read 1441792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.003425523, 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.048115779999999997, 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 7534 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490254+0000", 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490255+0000", 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490257+0000", 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4030000000000001e-06 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490264+0000", 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6039999999999998e-06 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.490269+0000", 2024-09-18T01:00:41.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536604+0000", 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046335227 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.536758+0000", 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015385400000000001 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.538370+0000", 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016120030000000001 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:71446 3.1c 3:39b29b41:::rbd_data.117c8d899572.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.92723505399999, 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046707666000000002, 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:41.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 71446 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566445+0000", 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566447+0000", 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1629999999999999e-06 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566459+0000", 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2173e-05 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566489+0000", 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0074e-05 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566504+0000", 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4696e-05 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566623+0000", 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119139 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.566761+0000", 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013833300000000001 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.575052+0000", 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082907699999999994 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612876+0000", 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037824296 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.612972+0000", 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6253000000000002e-05 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613071+0000", 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8968999999999994e-05 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:35.613152+0000", 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0799999999999999e-05 2024-09-18T01:00:41.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:137085 3.f 3:f2eb0918:::rbd_object_map.117c6de66219:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.516169958000006, 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94563719899999998, 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 137085 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977510+0000", 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977512+0000", 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.88e-06 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977522+0000", 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.007e-05 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977549+0000", 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7688999999999999e-05 2024-09-18T01:00:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977564+0000", 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4440000000000001e-05 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977734+0000", 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017047300000000001 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:41.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:16.977767+0000", 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2923000000000003e-05 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918540+0000", 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.94077240900000003 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.918717+0000", 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017784800000000001 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.919021+0000", 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030369099999999998 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.920184+0000", 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001163199 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923044+0000", 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0028592840000000001 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923105+0000", 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1132999999999999e-05 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:17.923147+0000", 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2160000000000003e-05 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:140635 3.d 3:b08054e4:::rbd_object_map.117ced55205b.0000000000000023:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:41.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.155056564000006, 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86222047999999996, 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 140635 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338623+0000", 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338625+0000", 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6199999999999999e-06 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338634+0000", 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9099999999999994e-06 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338657+0000", 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3672999999999998e-05 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338675+0000", 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7589999999999999e-05 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338767+0000", 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1896000000000002e-05 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:31.338776+0000", 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.38e-06 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200213+0000", 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.86143668200000001 2024-09-18T01:00:41.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200432+0000", 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021884199999999999 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:32.200844+0000", 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000411887 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:155664 3.d 3:b496760f:::rbd_data.117c1d78ece1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.587753460000002, 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046053647000000003, 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 155664 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.585 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905926+0000", 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.905927+0000", 2024-09-18T01:00:41.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.175e-06 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909267+0000", 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003339266 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909287+0000", 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0605e-05 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909297+0000", 2024-09-18T01:00:41.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3139999999999993e-06 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909363+0000", 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6437000000000001e-05 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909483+0000", 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119861 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-18T01:00:41.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.909868+0000", 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038564099999999999 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.917828+0000", 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079592570000000008 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951788+0000", 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.033960314999999998 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951895+0000", 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000107316 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:59:40.951980+0000", 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4460000000000001e-05 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:161321 3.d 3:b70a3213:::rbd_header.117c59271281:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e151)", 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:41.590 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.390114560999997, 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.84105173099999997, 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 161321 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103565+0000", 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103567+0000", 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.553e-06 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103576+0000", 2024-09-18T01:00:41.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4099999999999997e-06 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103601+0000", 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4833000000000001e-05 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103620+0000", 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9409999999999999e-05 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103789+0000", 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00016871599999999999 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.103807+0000", 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8502999999999999e-05 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.943761+0000", 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.83995325200000004 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.944023+0000", 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026232799999999999 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T01:00:09.944617+0000", 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00059372600000000004 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:41.596 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:41.597 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:00:41.598 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:41.592+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:41.698 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:41.696+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:41.768 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:41.768 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:41.769 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:41.769 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:41.782 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:00:41.830 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:41.828+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:41.900 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:41.900+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:41.902 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:41.902 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:41.902 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:41.903 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:41.903 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:41.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:00:41.927 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T01:00:42.001 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:42.000+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:42.070 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1061 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.076954+0000", 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.99034393299999, 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058909968, 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1061 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.076954+0000", 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077801+0000", 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084708899999999998 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077806+0000", 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1429999999999997e-06 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077816+0000", 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0003999999999999e-05 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077830+0000", 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4284000000000001e-05 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135677+0000", 2024-09-18T01:00:42.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.05784686 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135763+0000", 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.598e-05 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.135864+0000", 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100608 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1064 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077883+0000", 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.989414342, 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058942657000000002, 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1064 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077883+0000", 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077884+0000", 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077885+0000", 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4079999999999999e-06 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077890+0000", 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077893+0000", 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.534e-06 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136474+0000", 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058580656000000002 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136715+0000", 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024125199999999999 2024-09-18T01:00:42.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136826+0000", 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011095 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1065 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.077898+0000", 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.98940028199999, 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059188573000000001, 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1065 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.077898+0000", 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079036+0000", 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001137994 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079041+0000", 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1950000000000002e-06 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079053+0000", 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1895e-05 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079066+0000", 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3414000000000001e-05 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136866+0000", 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057800285 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.136948+0000", 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1366000000000007e-05 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.137086+0000", 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000138424 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1070 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.084 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079188+0000", 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.98810970700001, 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059023640000000002, 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1070 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T01:00:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079188+0000", 2024-09-18T01:00:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079189+0000", 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079190+0000", 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4339999999999999e-06 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079194+0000", 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4179999999999999e-06 2024-09-18T01:00:42.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079204+0000", 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3330000000000003e-06 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138009+0000", 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058805634000000002 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138099+0000", 2024-09-18T01:00:42.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9886e-05 2024-09-18T01:00:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138212+0000", 2024-09-18T01:00:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112488 2024-09-18T01:00:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1071 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1064960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.079209+0000", 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.98808923499999, 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059141978999999997, 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1071 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.079209+0000", 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081770+0000", 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025617650000000001 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081777+0000", 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8920000000000003e-06 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081787+0000", 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0066e-05 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.081794+0000", 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6000000000000003e-06 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138248+0000", 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056454273999999999 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138290+0000", 2024-09-18T01:00:42.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1603000000000001e-05 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.138351+0000", 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0779000000000002e-05 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1080 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:42.097 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.98529478200001, 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058826744, 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1080 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:42.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082003+0000", 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3099999999999999e-07 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082005+0000", 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8589999999999999e-06 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082010+0000", 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-06 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082014+0000", 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6270000000000002e-06 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140588+0000", 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058574185000000001 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140708+0000", 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011998999999999999 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140830+0000", 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000121902 2024-09-18T01:00:42.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1081 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1474560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082018+0000", 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.98527959800001, 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059065327000000001, 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1081 2024-09-18T01:00:42.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082018+0000", 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082019+0000", 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082020+0000", 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3829999999999999e-06 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082032+0000", 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2261e-05 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082036+0000", 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9820000000000002e-06 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140878+0000", 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058841633999999997 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.140966+0000", 2024-09-18T01:00:42.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8321999999999999e-05 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141084+0000", 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000117428 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1082 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.082041+0000", 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.985256652, 2024-09-18T01:00:42.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059359281999999999, 2024-09-18T01:00:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1082 2024-09-18T01:00:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T01:00:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.082041+0000", 2024-09-18T01:00:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085840+0000", 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037984160000000002 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085846+0000", 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1959999999999996e-06 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085857+0000", 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1127e-05 2024-09-18T01:00:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.085864+0000", 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1500000000000002e-06 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141139+0000", 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055274578999999997 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141276+0000", 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000137164 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.141400+0000", 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012464999999999999 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1097 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2129920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.981079013, 2024-09-18T01:00:42.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059056848000000002, 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1097 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086219+0000", 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0199999999999998e-07 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086221+0000", 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.488e-06 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086225+0000", 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4390000000000002e-06 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086234+0000", 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.918e-06 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145065+0000", 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058830907000000002 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145155+0000", 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0085000000000002e-05 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145276+0000", 2024-09-18T01:00:42.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120709 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1098 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.98105779599999, 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059298003000000002, 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1098 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086240+0000", 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1199999999999999e-07 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086242+0000", 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5120000000000001e-06 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086252+0000", 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7159999999999998e-06 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086257+0000", 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145325+0000", 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059068201000000001 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145418+0000", 2024-09-18T01:00:42.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2655e-05 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145538+0000", 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012036699999999999 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1099 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2211840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.086261+0000", 2024-09-18T01:00:42.120 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.981036413, 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059540516000000002, 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1099 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.086261+0000", 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089845+0000", 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003583578 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089851+0000", 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.784e-06 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089860+0000", 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-06 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.089867+0000", 2024-09-18T01:00:42.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6139999999999997e-06 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145585+0000", 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055717518000000001 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145676+0000", 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1020000000000006e-05 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.145802+0000", 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012630200000000001 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1112 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090161+0000", 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.977136559, 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058972203000000001, 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1112 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090161+0000", 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090162+0000", 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9800000000000004e-07 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090163+0000", 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4020000000000001e-06 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090168+0000", 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4580000000000003e-06 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090171+0000", 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4699999999999998e-06 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.148921+0000", 2024-09-18T01:00:42.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058750166999999999 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149014+0000", 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2689999999999995e-05 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149133+0000", 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000119518 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1113 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090175+0000", 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.977122358, 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059281499000000001, 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1113 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.130 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090175+0000", 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090176+0000", 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T01:00:42.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090180+0000", 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5630000000000002e-06 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090185+0000", 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7840000000000003e-06 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090189+0000", 2024-09-18T01:00:42.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7699999999999999e-06 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149194+0000", 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059005075999999997 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149286+0000", 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2448000000000007e-05 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149457+0000", 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000170562 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1114 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2834432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.97710432100001, 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059570682999999999, 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1114 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090194+0000", 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.96e-07 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090195+0000", 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.44e-06 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090200+0000", 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3610000000000003e-06 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090203+0000", 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4860000000000002e-06 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149512+0000", 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059308415000000003 2024-09-18T01:00:42.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149640+0000", 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012844899999999999 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149764+0000", 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012423600000000001 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1115 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.090208+0000", 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.97709025500001, 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.060040515000000003, 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1115 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.139 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.090208+0000", 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093900+0000", 2024-09-18T01:00:42.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003692252 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093906+0000", 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9159999999999998e-06 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093920+0000", 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4603000000000001e-05 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.093930+0000", 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3200000000000006e-06 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149804+0000", 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055874211999999999 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.149886+0000", 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1952000000000004e-05 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.150248+0000", 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00036225999999999999 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1129 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.97306490400001, 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058868049999999998, 2024-09-18T01:00:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1129 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094233+0000", 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094235+0000", 2024-09-18T01:00:42.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.57e-06 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094239+0000", 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3229999999999999e-06 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094243+0000", 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5279999999999999e-06 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.152924+0000", 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058681523999999999 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153015+0000", 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0529e-05 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153101+0000", 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6076999999999996e-05 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1130 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.973050892, 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059035721999999999, 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1130 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094247+0000", 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.84e-07 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094249+0000", 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4360000000000001e-06 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094253+0000", 2024-09-18T01:00:42.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094257+0000", 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6289999999999999e-06 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153145+0000", 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058888351999999998 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153219+0000", 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3780000000000002e-05 2024-09-18T01:00:42.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153283+0000", 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3829000000000002e-05 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:1131 3.16 3:6ad33745:::rbd_data.117c6d19cb58.0000000000000000:3 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:44.094261+0000", 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 177.973036928, 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059235327999999997, 2024-09-18T01:00:42.152 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 1131 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.094261+0000", 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097784+0000", 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0035226319999999999 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097791+0000", 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0690000000000004e-06 2024-09-18T01:00:42.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097806+0000", 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5277999999999998e-05 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.097817+0000", 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0628999999999999e-05 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153328+0000", 2024-09-18T01:00:42.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055511080999999997 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153413+0000", 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5010000000000001e-05 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:44.153496+0000", 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3628999999999995e-05 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4476.0:9366 3.2 e34/24 3:44bd9582:::rbd_data.117c918d6242.0000000000000000:head v 34'35, mlcod=34'33)", 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:57:48.856119+0000", 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 173.21117897400001, 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12572693800000001, 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T01:00:42.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856119+0000", 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.856122+0000", 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7970000000000001e-06 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901123+0000", 2024-09-18T01:00:42.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.045001698999999999 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901168+0000", 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5130000000000003e-05 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901188+0000", 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9029999999999999e-05 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901373+0000", 2024-09-18T01:00:42.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185918 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.901426+0000", 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2394e-05 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981713+0000", 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.080287391 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:57:48.981846+0000", 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013257899999999999 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:26734 3.16 3:6ffadde0:::rbd_data.117c4c7db1a0.0000000000000028:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:06.981595+0000", 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 155.08570260299999, 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058975160999999998, 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:00:42.161 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 26734 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981595+0000", 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981598+0000", 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7669999999999999e-06 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981612+0000", 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981647+0000", 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4666000000000003e-05 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981664+0000", 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6900000000000001e-05 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981824+0000", 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000160426 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.981964+0000", 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000139739 2024-09-18T01:00:42.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3", 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:06.994076+0000", 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012112206 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040436+0000", 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.046359848000000002 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040449+0000", 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2696e-05 2024-09-18T01:00:42.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040488+0000", 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8961000000000002e-05 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:07.040570+0000", 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2619000000000003e-05 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.166 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:42.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:00:42.168 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:42.164+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:42.269 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:42.268+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:42.343 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:42.343 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:42.343 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-18T01:00:42.343 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:42.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:00:42.369 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:42.368+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:42.470 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-18T01:00:42.468+0000 7fbc0b35c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26402) UID: 0 2024-09-18T01:00:42.477 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:42.477 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-18T01:00:42.477 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-18T01:00:42.477 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-18T01:00:42.477 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:42.492 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:00:42.570 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:42.568+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:42.665 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:00:42.665 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-18T01:00:42.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98052 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4179968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440036+0000", 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.22176485, 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131360583, 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98052 2024-09-18T01:00:42.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440036+0000", 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440037+0000", 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440040+0000", 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7159999999999999e-06 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440048+0000", 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7169999999999997e-06 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440056+0000", 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1860000000000003e-06 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.570989+0000", 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13093384999999999 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571175+0000", 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018504799999999999 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571397+0000", 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022245899999999999 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98053 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4466688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440064+0000", 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.22173698100001, 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13182195299999999, 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98053 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440064+0000", 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440065+0000", 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440068+0000", 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6699999999999998e-06 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440076+0000", 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7069999999999998e-06 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440083+0000", 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9929999999999998e-06 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571475+0000", 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13139158100000001 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571657+0000", 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018247099999999999 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571886+0000", 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022892400000000001 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98054 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2113536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440092+0000", 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.221709121, 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13228911600000001, 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98054 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440092+0000", 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440093+0000", 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440096+0000", 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440105+0000", 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7739999999999993e-06 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440112+0000", 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2130000000000002e-06 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.571970+0000", 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13185812199999999 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572159+0000", 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000188665 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572381+0000", 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022273200000000001 2024-09-18T01:00:42.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98055 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2400256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440122+0000", 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.22167894099999, 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273966600000001, 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98055 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440122+0000", 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440123+0000", 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3399999999999999e-07 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440126+0000", 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0029999999999999e-06 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440137+0000", 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0543000000000001e-05 2024-09-18T01:00:42.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440144+0000", 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572457+0000", 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132312807 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572636+0000", 2024-09-18T01:00:42.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017882299999999999 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572862+0000", 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022645200000000001 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98056 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2686976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440153+0000", 2024-09-18T01:00:42.683 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.221648364, 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13308801200000001, 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98056 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T01:00:42.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440153+0000", 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440154+0000", 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5300000000000004e-07 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440156+0000", 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.903e-06 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440165+0000", 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7600000000000008e-06 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440174+0000", 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6969999999999999e-06 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.572960+0000", 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132785984 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573087+0000", 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012728699999999999 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573241+0000", 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000153728 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98057 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2973696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440182+0000", 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.221618771, 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13357376900000001, 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98057 2024-09-18T01:00:42.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440182+0000", 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440183+0000", 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0299999999999999e-07 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440186+0000", 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8700000000000001e-06 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440195+0000", 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7099999999999996e-06 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440202+0000", 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2429999999999999e-06 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573338+0000", 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13313578400000001 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573526+0000", 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018780399999999999 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573756+0000", 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000230755 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98058 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440210+0000", 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.22159088799999, 2024-09-18T01:00:42.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13387749400000001, 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98058 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T01:00:42.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440210+0000", 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440627+0000", 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00041660699999999999 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440634+0000", 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.596e-06 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440650+0000", 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6164e-05 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440660+0000", 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.023e-05 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573833+0000", 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13317317000000001 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.573945+0000", 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000111731 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574088+0000", 2024-09-18T01:00:42.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000142996 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98059 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3547136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.440678+0000", 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.221123541, 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13391484000000001, 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98059 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.440678+0000", 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441760+0000", 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001082185 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441780+0000", 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0463e-05 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441829+0000", 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8763000000000001e-05 2024-09-18T01:00:42.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441849+0000", 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9423e-05 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574173+0000", 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13232460400000001 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574367+0000", 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019403000000000001 2024-09-18T01:00:42.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574593+0000", 2024-09-18T01:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000225372 2024-09-18T01:00:42.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98060 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3833856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441880+0000", 2024-09-18T01:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.21992145999999, 2024-09-18T01:00:42.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13320725899999999, 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98060 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441880+0000", 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441881+0000", 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6160000000000001e-06 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441889+0000", 2024-09-18T01:00:42.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8439999999999998e-06 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441909+0000", 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9856000000000001e-05 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441925+0000", 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6039999999999999e-05 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574669+0000", 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13274412799999999 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.574860+0000", 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019038599999999999 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575087+0000", 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022738899999999999 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.708 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98061 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4120576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.441947+0000", 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.219854704, 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13362664099999999, 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98061 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441947+0000", 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441948+0000", 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3200000000000001e-06 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441954+0000", 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3099999999999997e-06 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441981+0000", 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7095999999999999e-05 2024-09-18T01:00:42.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.441998+0000", 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6237000000000001e-05 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575165+0000", 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13316721200000001 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575355+0000", 2024-09-18T01:00:42.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000190218 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575573+0000", 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021824799999999999 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98062 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4407296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.442019+0000", 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.219782735, 2024-09-18T01:00:42.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13404179099999999, 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98062 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.442019+0000", 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445789+0000", 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037705569999999999 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445805+0000", 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.558e-05 2024-09-18T01:00:42.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445846+0000", 2024-09-18T01:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1066000000000003e-05 2024-09-18T01:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445871+0000", 2024-09-18T01:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5072999999999999e-05 2024-09-18T01:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575651+0000", 2024-09-18T01:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.12977999600000001 2024-09-18T01:00:42.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.575835+0000", 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000183931 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576060+0000", 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022558800000000001 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98064 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2115584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.445983+0000", 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.21581829599999, 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13103894799999999, 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98064 2024-09-18T01:00:42.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445983+0000", 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445985+0000", 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9530000000000002e-06 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.445991+0000", 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446014+0000", 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3056000000000001e-05 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446030+0000", 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5662999999999999e-05 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576622+0000", 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13059273900000001 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.576805+0000", 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018304899999999999 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577022+0000", 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000216481 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98065 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2402304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446050+0000", 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.21575118, 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.131457463, 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.723 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98065 2024-09-18T01:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446050+0000", 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446051+0000", 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1260000000000001e-06 2024-09-18T01:00:42.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446057+0000", 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2040000000000002e-06 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446077+0000", 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9766e-05 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446092+0000", 2024-09-18T01:00:42.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5124e-05 2024-09-18T01:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577098+0000", 2024-09-18T01:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13100595100000001 2024-09-18T01:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577284+0000", 2024-09-18T01:00:42.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000186152 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577508+0000", 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022314000000000001 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98066 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2689024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446112+0000", 2024-09-18T01:00:42.728 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.215688974, 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13189522100000001, 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98066 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446112+0000", 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446113+0000", 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0240000000000001e-06 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446120+0000", 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446137+0000", 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7479e-05 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446152+0000", 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4633e-05 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577587+0000", 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13143572100000001 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.577786+0000", 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019872999999999999 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578008+0000", 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022141399999999999 2024-09-18T01:00:42.732 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98067 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 2975744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446170+0000", 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.215631311, 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132319567, 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.733 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98067 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446170+0000", 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446171+0000", 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0860000000000001e-06 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446178+0000", 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6699999999999997e-06 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446196+0000", 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7989999999999999e-05 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446213+0000", 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7493999999999999e-05 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578083+0000", 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13186995000000001 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578266+0000", 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018321099999999999 2024-09-18T01:00:42.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578490+0000", 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022316600000000001 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98068 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3262464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446231+0000", 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.21557040499999, 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13273661, 2024-09-18T01:00:42.737 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98068 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446231+0000", 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446232+0000", 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.187e-06 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446238+0000", 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0070000000000003e-06 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446257+0000", 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8732999999999998e-05 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446272+0000", 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5532999999999998e-05 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578569+0000", 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.132296631 2024-09-18T01:00:42.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578749+0000", 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017978300000000001 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.578967+0000", 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000218736 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98069 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3549184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446291+0000", 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.215510732, 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13311771, 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98069 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.742 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446291+0000", 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446292+0000", 2024-09-18T01:00:42.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4529999999999999e-06 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446298+0000", 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4470000000000003e-06 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446340+0000", 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1705999999999997e-05 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446358+0000", 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7873999999999999e-05 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579072+0000", 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13271396399999999 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579213+0000", 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014133900000000001 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579408+0000", 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019492700000000001 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98070 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 3835904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446384+0000", 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.215417289, 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133517145, 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98070 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446384+0000", 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446385+0000", 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.37e-06 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446393+0000", 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2629999999999997e-06 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446413+0000", 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9930000000000001e-05 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446428+0000", 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5336e-05 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579494+0000", 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13306637399999999 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579682+0000", 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018740900000000001 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579901+0000", 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021946300000000001 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98071 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.750 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.446447+0000", 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.215354123, 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.133931615, 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98071 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T01:00:42.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.446447+0000", 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449904+0000", 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003456822 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449945+0000", 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0896000000000003e-05 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449971+0000", 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5996999999999999e-05 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.449986+0000", 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.49e-05 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.579978+0000", 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.129992723 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580157+0000", 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000179006 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580379+0000", 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022127099999999999 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4476.0:98073 3.1d 3:bbf75b98:::rbd_data.117c304cc916.0000000000000001:1e [read 4696064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-18T00:58:54.450056+0000", 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.21174499599999, 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.13113503100000001, 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4476", 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/4241641732", 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 98073 2024-09-18T01:00:42.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450056+0000", 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450057+0000", 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8700000000000005e-07 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450060+0000", 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5329999999999999e-06 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450072+0000", 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1919999999999999e-05 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.450080+0000", 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9500000000000001e-06 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.580951+0000", 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.130871128 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581057+0000", 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010501399999999999 2024-09-18T01:00:42.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-18T01:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-18T01:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-18T01:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-18T00:58:54.581191+0000", 2024-09-18T01:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013479900000000001 2024-09-18T01:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-18T01:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-18T01:00:42.759 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:00:42.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:00:42.761 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:42.760+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:42.761 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:00:42.862 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:42.860+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:42.893 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-18T01:00:42.894 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-18T01:00:42.894 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-18T01:00:42.894 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-18T01:00:42.909 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:00:42.959 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-18T01:00:42.959 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:00:42.962 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:42.960+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:42.975 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-18T01:00:42.975 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:00:42.992 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-18T01:00:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:00:43.010 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-18T01:00:43.010 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:00:43.028 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-18T01:00:43.029 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:00:43.029 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:43.029 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:43.043 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:00:43.049 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-18T01:00:43.049 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:00:43.063 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:43.060+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:43.069 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-18T01:00:43.069 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T01:00:43.088 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-18T01:00:43.089 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T01:00:43.166 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:43.164+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:43.177 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:43.177 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:43.193 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:00:43.280 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:43.280+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:43.358 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:43.358 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:43.367 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:43.364+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:43.373 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:00:43.468 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:43.464+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:43.544 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:43.544 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:43.558 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:00:43.604 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:43.604+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:43.669 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-18T01:00:43.668+0000 7f9e6e904640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26591) UID: 0 2024-09-18T01:00:43.680 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:43.680 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:43.695 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T01:00:43.770 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-18T01:00:43.768+0000 7f92baa54640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26398) UID: 0 2024-09-18T01:00:43.859 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:43.860 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:43.871 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:43.868+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:43.874 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T01:00:43.960 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:00:43.972 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-18T01:00:43.972+0000 7faf518e3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26589) UID: 0 2024-09-18T01:00:44.042 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:44.042 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:44.056 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T01:00:44.074 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-18T01:00:44.072+0000 7ff28b9e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26400) UID: 0 2024-09-18T01:00:44.174 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-18T01:00:44.172+0000 7f77f5d60640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26396) UID: 0 2024-09-18T01:00:44.230 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:44.230 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:44.245 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T01:00:44.260 INFO:tasks.thrashosds.thrasher:in_osds: [1, 4, 5, 6, 7, 3] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T01:00:44.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:00:44.283 INFO:teuthology.orchestra.run.smithi042.stdout:All operations completed A-OK! 2024-09-18T01:00:44.291 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-18T01:00:44.292+0000 7fb8c4d8e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26586) UID: 0 2024-09-18T01:00:44.338 DEBUG:teuthology.parallel:result is None 2024-09-18T01:00:44.339 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-18T01:00:44.348 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-18T01:00:44.356 INFO:tasks.thrashosds:joining thrashosds 2024-09-18T01:00:44.366 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:44.366 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:44.381 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T01:00:44.550 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:44.550 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:44.566 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T01:00:44.690 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-18T01:00:44.690 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:00:45.151 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T01:00:45.152 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:00:45.181 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:00:46.154 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:00:46.183 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T01:00:46.929 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T01:00:46.962 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T01:00:47.178 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:00:47.932 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T01:00:47.937 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-18T01:00:47.962 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T01:00:47.963 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:00:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-18T01:00:49.174 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T01:00:49.205 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:00:50.177 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:00:50.201 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:00:50.202 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:00:50.504 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:00:50.505 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T01:00:50.523 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-18T01:00:50.173306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":482,"num_read":220711,"num_read_kb":1067392,"num_write":22422,"num_write_kb":690995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9854,"num_bytes_recovered":693846218,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19658,"ondisk_log_size":19658,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":309828,"kb_used_data":14376,"kb_used_omap":9,"kb_used_meta":295286,"kb_avail":565921212,"statfs":{"total":579820584960,"available":579503321088,"internally_reserved":0,"allocated":14721024,"data_stored":7687685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":302373542},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016185"},"pg_stats":[{"pgid":"3.1f","version":"186'621","reported_seq":4171,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572122+0000","last_change":"2024-09-18T01:00:44.155176+0000","last_active":"2024-09-18T01:00:45.572122+0000","last_peered":"2024-09-18T01:00:45.572122+0000","last_clean":"2024-09-18T01:00:45.572122+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T01:00:45.572122+0000","last_undegraded":"2024-09-18T01:00:45.572122+0000","last_fullsized":"2024-09-18T01:00:45.572122+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:49.598474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001427074,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6437,"num_read_kb":23165,"num_write":614,"num_write_kb":11006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.1e","version":"186'646","reported_seq":5444,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572219+0000","last_change":"2024-09-18T01:00:44.158389+0000","last_active":"2024-09-18T01:00:45.572219+0000","last_peered":"2024-09-18T01:00:45.572219+0000","last_clean":"2024-09-18T01:00:45.572219+0000","last_became_active":"2024-09-18T01:00:23.947401+0000","last_became_peered":"2024-09-18T01:00:23.947401+0000","last_unstale":"2024-09-18T01:00:45.572219+0000","last_undegraded":"2024-09-18T01:00:45.572219+0000","last_fullsized":"2024-09-18T01:00:45.572219+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:39:19.159376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041620060000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4822,"num_read_kb":36392,"num_write":703,"num_write_kb":11559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":30825045,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.1d","version":"184'800","reported_seq":12660,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.154432+0000","last_change":"2024-09-18T01:00:44.147606+0000","last_active":"2024-09-18T01:00:45.154432+0000","last_peered":"2024-09-18T01:00:45.154432+0000","last_clean":"2024-09-18T01:00:45.154432+0000","last_became_active":"2024-09-18T01:00:20.005875+0000","last_became_peered":"2024-09-18T01:00:20.005875+0000","last_unstale":"2024-09-18T01:00:45.154432+0000","last_undegraded":"2024-09-18T01:00:45.154432+0000","last_fullsized":"2024-09-18T01:00:45.154432+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:08.761605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034116569999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":13334,"num_read_kb":40113,"num_write":945,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":22094834,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.1c","version":"186'663","reported_seq":4532,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.709971+0000","last_change":"2024-09-18T01:00:44.151679+0000","last_active":"2024-09-18T01:00:45.709971+0000","last_peered":"2024-09-18T01:00:45.709971+0000","last_clean":"2024-09-18T01:00:45.709971+0000","last_became_active":"2024-09-18T00:59:17.918223+0000","last_became_peered":"2024-09-18T00:59:17.918223+0000","last_unstale":"2024-09-18T01:00:45.709971+0000","last_undegraded":"2024-09-18T01:00:45.709971+0000","last_fullsized":"2024-09-18T01:00:45.709971+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:30:03.649520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002343262,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5693,"num_read_kb":28138,"num_write":736,"num_write_kb":11405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":5428683,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.1b","version":"186'1093","reported_seq":10545,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156312+0000","last_change":"2024-09-18T01:00:44.149683+0000","last_active":"2024-09-18T01:00:45.156312+0000","last_peered":"2024-09-18T01:00:45.156312+0000","last_clean":"2024-09-18T01:00:45.156312+0000","last_became_active":"2024-09-18T01:00:22.941911+0000","last_became_peered":"2024-09-18T01:00:22.941911+0000","last_unstale":"2024-09-18T01:00:45.156312+0000","last_undegraded":"2024-09-18T01:00:45.156312+0000","last_fullsized":"2024-09-18T01:00:45.156312+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:31:18.554987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003279527,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12686,"num_read_kb":40787,"num_write":1375,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":495,"num_bytes_recovered":59340116,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.1a","version":"186'364","reported_seq":5092,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.709995+0000","last_change":"2024-09-18T01:00:44.151068+0000","last_active":"2024-09-18T01:00:45.709995+0000","last_peered":"2024-09-18T01:00:45.709995+0000","last_clean":"2024-09-18T01:00:45.709995+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T01:00:45.709995+0000","last_undegraded":"2024-09-18T01:00:45.709995+0000","last_fullsized":"2024-09-18T01:00:45.709995+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:24.750570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017298630000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5410,"num_read_kb":29581,"num_write":319,"num_write_kb":12768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.19","version":"186'701","reported_seq":30835,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.709738+0000","last_change":"2024-09-18T01:00:44.150671+0000","last_active":"2024-09-18T01:00:45.709738+0000","last_peered":"2024-09-18T01:00:45.709738+0000","last_clean":"2024-09-18T01:00:45.709738+0000","last_became_active":"2024-09-18T01:00:10.952126+0000","last_became_peered":"2024-09-18T01:00:10.952126+0000","last_unstale":"2024-09-18T01:00:45.709738+0000","last_undegraded":"2024-09-18T01:00:45.709738+0000","last_fullsized":"2024-09-18T01:00:45.709738+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:20:32.250982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001459178,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":30614,"num_read_kb":78260,"num_write":886,"num_write_kb":27079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":40291284,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.18","version":"186'656","reported_seq":3965,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572006+0000","last_change":"2024-09-18T01:00:44.157985+0000","last_active":"2024-09-18T01:00:45.572006+0000","last_peered":"2024-09-18T01:00:45.572006+0000","last_clean":"2024-09-18T01:00:45.572006+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T01:00:45.572006+0000","last_undegraded":"2024-09-18T01:00:45.572006+0000","last_fullsized":"2024-09-18T01:00:45.572006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:27.075227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039694869999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4869,"num_read_kb":13814,"num_write":877,"num_write_kb":42194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.17","version":"185'676","reported_seq":7303,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572211+0000","last_change":"2024-09-18T01:00:44.156705+0000","last_active":"2024-09-18T01:00:45.572211+0000","last_peered":"2024-09-18T01:00:45.572211+0000","last_clean":"2024-09-18T01:00:45.572211+0000","last_became_active":"2024-09-18T01:00:20.017711+0000","last_became_peered":"2024-09-18T01:00:20.017711+0000","last_unstale":"2024-09-18T01:00:45.572211+0000","last_undegraded":"2024-09-18T01:00:45.572211+0000","last_fullsized":"2024-09-18T01:00:45.572211+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:49.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028584439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7284,"num_read_kb":46982,"num_write":773,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":13346337,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.16","version":"186'641","reported_seq":8661,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.571866+0000","last_change":"2024-09-18T01:00:44.155480+0000","last_active":"2024-09-18T01:00:45.571866+0000","last_peered":"2024-09-18T01:00:45.571866+0000","last_clean":"2024-09-18T01:00:45.571866+0000","last_became_active":"2024-09-18T01:00:31.070442+0000","last_became_peered":"2024-09-18T01:00:31.070442+0000","last_unstale":"2024-09-18T01:00:45.571866+0000","last_undegraded":"2024-09-18T01:00:45.571866+0000","last_fullsized":"2024-09-18T01:00:45.571866+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:43:37.754762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887849999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8507,"num_read_kb":37252,"num_write":699,"num_write_kb":22150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":37601010,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.15","version":"184'519","reported_seq":2989,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.261870+0000","last_change":"2024-09-18T01:00:44.262860+0000","last_active":"2024-09-18T01:00:44.261870+0000","last_peered":"2024-09-18T01:00:44.261870+0000","last_clean":"2024-09-18T01:00:44.261870+0000","last_became_active":"2024-09-18T01:00:21.007605+0000","last_became_peered":"2024-09-18T01:00:21.007605+0000","last_unstale":"2024-09-18T01:00:44.261870+0000","last_undegraded":"2024-09-18T01:00:44.261870+0000","last_fullsized":"2024-09-18T01:00:44.261870+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:20:41.796582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035212300000000002,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2322,"num_read_kb":15300,"num_write":809,"num_write_kb":48070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":22291504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.14","version":"181'371","reported_seq":2973,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156375+0000","last_change":"2024-09-18T01:00:44.147456+0000","last_active":"2024-09-18T01:00:45.156375+0000","last_peered":"2024-09-18T01:00:45.156375+0000","last_clean":"2024-09-18T01:00:45.156375+0000","last_became_active":"2024-09-18T01:00:31.066969+0000","last_became_peered":"2024-09-18T01:00:31.066969+0000","last_unstale":"2024-09-18T01:00:45.156375+0000","last_undegraded":"2024-09-18T01:00:45.156375+0000","last_fullsized":"2024-09-18T01:00:45.156375+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:25:29.555120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013361650000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2452,"num_read_kb":3816,"num_write":339,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":469,"num_bytes_recovered":3474495,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.154075+0000","last_change":"2024-09-18T01:00:15.995790+0000","last_active":"2024-09-18T01:00:45.154075+0000","last_peered":"2024-09-18T01:00:45.154075+0000","last_clean":"2024-09-18T01:00:45.154075+0000","last_became_active":"2024-09-18T01:00:15.995425+0000","last_became_peered":"2024-09-18T01:00:15.995425+0000","last_unstale":"2024-09-18T01:00:45.154075+0000","last_undegraded":"2024-09-18T01:00:45.154075+0000","last_fullsized":"2024-09-18T01:00:45.154075+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:49.875513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"186'777","reported_seq":4829,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156357+0000","last_change":"2024-09-18T01:00:44.148713+0000","last_active":"2024-09-18T01:00:45.156357+0000","last_peered":"2024-09-18T01:00:45.156357+0000","last_clean":"2024-09-18T01:00:45.156357+0000","last_became_active":"2024-09-18T01:00:11.936218+0000","last_became_peered":"2024-09-18T01:00:11.936218+0000","last_unstale":"2024-09-18T01:00:45.156357+0000","last_undegraded":"2024-09-18T01:00:45.156357+0000","last_fullsized":"2024-09-18T01:00:45.156357+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:24.939297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002351681,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5488,"num_read_kb":22949,"num_write":1002,"num_write_kb":4650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":2805113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":353,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156369+0000","last_change":"2024-09-18T00:59:15.073941+0000","last_active":"2024-09-18T01:00:45.156369+0000","last_peered":"2024-09-18T01:00:45.156369+0000","last_clean":"2024-09-18T01:00:45.156369+0000","last_became_active":"2024-09-18T00:59:15.072986+0000","last_became_peered":"2024-09-18T00:59:15.072986+0000","last_unstale":"2024-09-18T01:00:45.156369+0000","last_undegraded":"2024-09-18T01:00:45.156369+0000","last_fullsized":"2024-09-18T01:00:45.156369+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:16.671308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"185'506","reported_seq":8829,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156623+0000","last_change":"2024-09-18T01:00:44.150561+0000","last_active":"2024-09-18T01:00:45.156623+0000","last_peered":"2024-09-18T01:00:45.156623+0000","last_clean":"2024-09-18T01:00:45.156623+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T01:00:45.156623+0000","last_undegraded":"2024-09-18T01:00:45.156623+0000","last_fullsized":"2024-09-18T01:00:45.156623+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:44.745709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036716330000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8918,"num_read_kb":44936,"num_write":410,"num_write_kb":13864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.154571+0000","last_change":"2024-09-18T01:00:15.991240+0000","last_active":"2024-09-18T01:00:45.154571+0000","last_peered":"2024-09-18T01:00:45.154571+0000","last_clean":"2024-09-18T01:00:45.154571+0000","last_became_active":"2024-09-18T01:00:15.990930+0000","last_became_peered":"2024-09-18T01:00:15.990930+0000","last_unstale":"2024-09-18T01:00:45.154571+0000","last_undegraded":"2024-09-18T01:00:45.154571+0000","last_fullsized":"2024-09-18T01:00:45.154571+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:55.616429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"185'348","reported_seq":4436,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.260079+0000","last_change":"2024-09-18T01:00:44.260854+0000","last_active":"2024-09-18T01:00:44.260079+0000","last_peered":"2024-09-18T01:00:44.260079+0000","last_clean":"2024-09-18T01:00:44.260079+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T01:00:44.260079+0000","last_undegraded":"2024-09-18T01:00:44.260079+0000","last_fullsized":"2024-09-18T01:00:44.260079+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:09.012851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019390989999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3644,"num_read_kb":27224,"num_write":289,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":347,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572005+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T01:00:45.572005+0000","last_peered":"2024-09-18T01:00:45.572005+0000","last_clean":"2024-09-18T01:00:45.572005+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T01:00:45.572005+0000","last_undegraded":"2024-09-18T01:00:45.572005+0000","last_fullsized":"2024-09-18T01:00:45.572005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"185'580","reported_seq":4466,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.260943+0000","last_change":"2024-09-18T01:00:44.261780+0000","last_active":"2024-09-18T01:00:44.260943+0000","last_peered":"2024-09-18T01:00:44.260943+0000","last_clean":"2024-09-18T01:00:44.260943+0000","last_became_active":"2024-09-18T00:58:58.963883+0000","last_became_peered":"2024-09-18T00:58:58.963883+0000","last_unstale":"2024-09-18T01:00:44.260943+0000","last_undegraded":"2024-09-18T01:00:44.260943+0000","last_fullsized":"2024-09-18T01:00:44.260943+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:34:48.165282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027378979999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4830,"num_read_kb":33562,"num_write":800,"num_write_kb":24925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.3","version":"186'946","reported_seq":6113,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156120+0000","last_change":"2024-09-18T01:00:44.148549+0000","last_active":"2024-09-18T01:00:45.156120+0000","last_peered":"2024-09-18T01:00:45.156120+0000","last_clean":"2024-09-18T01:00:45.156120+0000","last_became_active":"2024-09-18T01:00:21.926403+0000","last_became_peered":"2024-09-18T01:00:21.926403+0000","last_unstale":"2024-09-18T01:00:45.156120+0000","last_undegraded":"2024-09-18T01:00:45.156120+0000","last_fullsized":"2024-09-18T01:00:45.156120+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:09:46.795184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022401439999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7645,"num_read_kb":19179,"num_write":1076,"num_write_kb":33944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":99850853,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.2","version":"23'2","reported_seq":204,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.259234+0000","last_change":"2024-09-18T01:00:28.086666+0000","last_active":"2024-09-18T01:00:44.259234+0000","last_peered":"2024-09-18T01:00:44.259234+0000","last_clean":"2024-09-18T01:00:44.259234+0000","last_became_active":"2024-09-18T01:00:28.078264+0000","last_became_peered":"2024-09-18T01:00:28.078264+0000","last_unstale":"2024-09-18T01:00:44.259234+0000","last_undegraded":"2024-09-18T01:00:44.259234+0000","last_fullsized":"2024-09-18T01:00:44.259234+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:15.282919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"185'657","reported_seq":8330,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.571628+0000","last_change":"2024-09-18T01:00:44.156905+0000","last_active":"2024-09-18T01:00:45.571628+0000","last_peered":"2024-09-18T01:00:45.571628+0000","last_clean":"2024-09-18T01:00:45.571628+0000","last_became_active":"2024-09-18T01:00:08.951992+0000","last_became_peered":"2024-09-18T01:00:08.951992+0000","last_unstale":"2024-09-18T01:00:45.571628+0000","last_undegraded":"2024-09-18T01:00:45.571628+0000","last_fullsized":"2024-09-18T01:00:45.571628+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:59.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030555019999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7663,"num_read_kb":29924,"num_write":748,"num_write_kb":19563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":15190015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":95,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-18T01:00:47.938432+0000","last_change":"2024-09-18T01:00:04.976725+0000","last_active":"2024-09-18T01:00:47.938432+0000","last_peered":"2024-09-18T01:00:47.938432+0000","last_clean":"2024-09-18T01:00:47.938432+0000","last_became_active":"2024-09-18T01:00:04.976360+0000","last_became_peered":"2024-09-18T01:00:04.976360+0000","last_unstale":"2024-09-18T01:00:47.938432+0000","last_undegraded":"2024-09-18T01:00:47.938432+0000","last_fullsized":"2024-09-18T01:00:47.938432+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:11.159126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"184'548","reported_seq":9364,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.259969+0000","last_change":"2024-09-18T01:00:44.261168+0000","last_active":"2024-09-18T01:00:44.259969+0000","last_peered":"2024-09-18T01:00:44.259969+0000","last_clean":"2024-09-18T01:00:44.259969+0000","last_became_active":"2024-09-18T01:00:31.069677+0000","last_became_peered":"2024-09-18T01:00:31.069677+0000","last_unstale":"2024-09-18T01:00:44.259969+0000","last_undegraded":"2024-09-18T01:00:44.259969+0000","last_fullsized":"2024-09-18T01:00:44.259969+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:38:57.075708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726180000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8919,"num_read_kb":48544,"num_write":559,"num_write_kb":18757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":794,"num_bytes_recovered":19034719,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.153462+0000","last_change":"2024-09-18T01:00:15.994517+0000","last_active":"2024-09-18T01:00:45.153462+0000","last_peered":"2024-09-18T01:00:45.153462+0000","last_clean":"2024-09-18T01:00:45.153462+0000","last_became_active":"2024-09-18T01:00:15.994159+0000","last_became_peered":"2024-09-18T01:00:15.994159+0000","last_unstale":"2024-09-18T01:00:45.153462+0000","last_undegraded":"2024-09-18T01:00:45.153462+0000","last_fullsized":"2024-09-18T01:00:45.153462+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:06.775713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"186'722","reported_seq":5041,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156024+0000","last_change":"2024-09-18T01:00:44.147483+0000","last_active":"2024-09-18T01:00:45.156024+0000","last_peered":"2024-09-18T01:00:45.156024+0000","last_clean":"2024-09-18T01:00:45.156024+0000","last_became_active":"2024-09-18T00:59:18.918527+0000","last_became_peered":"2024-09-18T00:59:18.918527+0000","last_unstale":"2024-09-18T01:00:45.156024+0000","last_undegraded":"2024-09-18T01:00:45.156024+0000","last_fullsized":"2024-09-18T01:00:45.156024+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:31.958835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00139405,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5240,"num_read_kb":16439,"num_write":1146,"num_write_kb":52135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":12961092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.3","version":"21'1","reported_seq":81,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.153959+0000","last_change":"2024-09-18T01:00:28.080162+0000","last_active":"2024-09-18T01:00:45.153959+0000","last_peered":"2024-09-18T01:00:45.153959+0000","last_clean":"2024-09-18T01:00:45.153959+0000","last_became_active":"2024-09-18T01:00:28.075757+0000","last_became_peered":"2024-09-18T01:00:28.075757+0000","last_unstale":"2024-09-18T01:00:45.153959+0000","last_undegraded":"2024-09-18T01:00:45.153959+0000","last_fullsized":"2024-09-18T01:00:45.153959+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:29.327404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":499,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.709650+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T01:00:45.709650+0000","last_peered":"2024-09-18T01:00:45.709650+0000","last_clean":"2024-09-18T01:00:45.709650+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T01:00:45.709650+0000","last_undegraded":"2024-09-18T01:00:45.709650+0000","last_fullsized":"2024-09-18T01:00:45.709650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"184'448","reported_seq":5825,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156006+0000","last_change":"2024-09-18T01:00:44.149901+0000","last_active":"2024-09-18T01:00:45.156006+0000","last_peered":"2024-09-18T01:00:45.156006+0000","last_clean":"2024-09-18T01:00:45.156006+0000","last_became_active":"2024-09-18T01:00:08.949384+0000","last_became_peered":"2024-09-18T01:00:08.949384+0000","last_unstale":"2024-09-18T01:00:45.156006+0000","last_undegraded":"2024-09-18T01:00:45.156006+0000","last_fullsized":"2024-09-18T01:00:45.156006+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:20.064612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033952990000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5722,"num_read_kb":51790,"num_write":458,"num_write_kb":39546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":32435091,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.9","version":"185'408","reported_seq":3728,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.153549+0000","last_change":"2024-09-18T01:00:44.145255+0000","last_active":"2024-09-18T01:00:45.153549+0000","last_peered":"2024-09-18T01:00:45.153549+0000","last_clean":"2024-09-18T01:00:45.153549+0000","last_became_active":"2024-09-18T01:00:18.997710+0000","last_became_peered":"2024-09-18T01:00:18.997710+0000","last_unstale":"2024-09-18T01:00:45.153549+0000","last_undegraded":"2024-09-18T01:00:45.153549+0000","last_fullsized":"2024-09-18T01:00:45.153549+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:36.454597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016382809999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3625,"num_read_kb":11553,"num_write":342,"num_write_kb":23969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":27148173,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.a","version":"185'607","reported_seq":5574,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.153657+0000","last_change":"2024-09-18T01:00:44.146011+0000","last_active":"2024-09-18T01:00:45.153657+0000","last_peered":"2024-09-18T01:00:45.153657+0000","last_clean":"2024-09-18T01:00:45.153657+0000","last_became_active":"2024-09-18T01:00:22.938344+0000","last_became_peered":"2024-09-18T01:00:22.938344+0000","last_unstale":"2024-09-18T01:00:45.153657+0000","last_undegraded":"2024-09-18T01:00:45.153657+0000","last_fullsized":"2024-09-18T01:00:45.153657+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:59:54.570595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021924589999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4754,"num_read_kb":46819,"num_write":596,"num_write_kb":17196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":6639875,"num_keys_recovered":85,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.b","version":"185'515","reported_seq":5867,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-18T01:00:47.938035+0000","last_change":"2024-09-18T01:00:44.262988+0000","last_active":"2024-09-18T01:00:47.938035+0000","last_peered":"2024-09-18T01:00:47.938035+0000","last_clean":"2024-09-18T01:00:47.938035+0000","last_became_active":"2024-09-18T01:00:10.954910+0000","last_became_peered":"2024-09-18T01:00:10.954910+0000","last_unstale":"2024-09-18T01:00:47.938035+0000","last_undegraded":"2024-09-18T01:00:47.938035+0000","last_fullsized":"2024-09-18T01:00:47.938035+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:45.069175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042875949999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5466,"num_read_kb":66445,"num_write":496,"num_write_kb":15031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15399944,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"4"}]},{"pgid":"3.c","version":"185'566","reported_seq":7089,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-18T01:00:47.938077+0000","last_change":"2024-09-18T01:00:44.261560+0000","last_active":"2024-09-18T01:00:47.938077+0000","last_peered":"2024-09-18T01:00:47.938077+0000","last_clean":"2024-09-18T01:00:47.938077+0000","last_became_active":"2024-09-18T01:00:19.999750+0000","last_became_peered":"2024-09-18T01:00:19.999750+0000","last_unstale":"2024-09-18T01:00:47.938077+0000","last_undegraded":"2024-09-18T01:00:47.938077+0000","last_fullsized":"2024-09-18T01:00:47.938077+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:23.055801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030770020000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6401,"num_read_kb":47394,"num_write":633,"num_write_kb":22377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":26783458,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"4"}]},{"pgid":"3.d","version":"184'976","reported_seq":6750,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.154283+0000","last_change":"2024-09-18T01:00:44.146438+0000","last_active":"2024-09-18T01:00:45.154283+0000","last_peered":"2024-09-18T01:00:45.154283+0000","last_clean":"2024-09-18T01:00:45.154283+0000","last_became_active":"2024-09-18T01:00:21.923279+0000","last_became_peered":"2024-09-18T01:00:21.923279+0000","last_unstale":"2024-09-18T01:00:45.154283+0000","last_undegraded":"2024-09-18T01:00:45.154283+0000","last_fullsized":"2024-09-18T01:00:45.154283+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:23.093467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026100659999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8636,"num_read_kb":40955,"num_write":1143,"num_write_kb":30743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":80305919,"num_keys_recovered":163,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.e","version":"183'713","reported_seq":4734,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572023+0000","last_change":"2024-09-18T01:00:44.157454+0000","last_active":"2024-09-18T01:00:45.572023+0000","last_peered":"2024-09-18T01:00:45.572023+0000","last_clean":"2024-09-18T01:00:45.572023+0000","last_became_active":"2024-09-18T01:00:21.008903+0000","last_became_peered":"2024-09-18T01:00:21.008903+0000","last_unstale":"2024-09-18T01:00:45.572023+0000","last_undegraded":"2024-09-18T01:00:45.572023+0000","last_fullsized":"2024-09-18T01:00:45.572023+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:53.068038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034902890000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4741,"num_read_kb":29100,"num_write":948,"num_write_kb":26466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":41587165,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.f","version":"184'563","reported_seq":4323,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.710171+0000","last_change":"2024-09-18T01:00:44.152013+0000","last_active":"2024-09-18T01:00:45.710171+0000","last_peered":"2024-09-18T01:00:45.710171+0000","last_clean":"2024-09-18T01:00:45.710171+0000","last_became_active":"2024-09-18T00:59:17.917905+0000","last_became_peered":"2024-09-18T00:59:17.917905+0000","last_unstale":"2024-09-18T01:00:45.710171+0000","last_undegraded":"2024-09-18T01:00:45.710171+0000","last_fullsized":"2024-09-18T01:00:45.710171+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:51:06.807009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024190230000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5311,"num_read_kb":28191,"num_write":598,"num_write_kb":9287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":6320640,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.10","version":"184'456","reported_seq":5250,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.709630+0000","last_change":"2024-09-18T01:00:44.148986+0000","last_active":"2024-09-18T01:00:45.709630+0000","last_peered":"2024-09-18T01:00:45.709630+0000","last_clean":"2024-09-18T01:00:45.709630+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T01:00:45.709630+0000","last_undegraded":"2024-09-18T01:00:45.709630+0000","last_fullsized":"2024-09-18T01:00:45.709630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:48:20.609462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001443239,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":51584,"num_write":357,"num_write_kb":26802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.11","version":"183'292","reported_seq":3191,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.153805+0000","last_change":"2024-09-18T01:00:44.147261+0000","last_active":"2024-09-18T01:00:45.153805+0000","last_peered":"2024-09-18T01:00:45.153805+0000","last_clean":"2024-09-18T01:00:45.153805+0000","last_became_active":"2024-09-18T01:00:19.002365+0000","last_became_peered":"2024-09-18T01:00:19.002365+0000","last_unstale":"2024-09-18T01:00:45.153805+0000","last_undegraded":"2024-09-18T01:00:45.153805+0000","last_fullsized":"2024-09-18T01:00:45.153805+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:37:21.382231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033253200000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1897,"num_read_kb":13175,"num_write":246,"num_write_kb":14260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":15824068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.12","version":"184'638","reported_seq":6854,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.261237+0000","last_change":"2024-09-18T01:00:44.261997+0000","last_active":"2024-09-18T01:00:44.261237+0000","last_peered":"2024-09-18T01:00:44.261237+0000","last_clean":"2024-09-18T01:00:44.261237+0000","last_became_active":"2024-09-18T01:00:21.926814+0000","last_became_peered":"2024-09-18T01:00:21.926814+0000","last_unstale":"2024-09-18T01:00:44.261237+0000","last_undegraded":"2024-09-18T01:00:44.261237+0000","last_fullsized":"2024-09-18T01:00:44.261237+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:04.145432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002915932,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6954,"num_read_kb":17130,"num_write":754,"num_write_kb":11285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":32804395,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.13","version":"185'521","reported_seq":4574,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.154187+0000","last_change":"2024-09-18T01:00:44.148400+0000","last_active":"2024-09-18T01:00:45.154187+0000","last_peered":"2024-09-18T01:00:45.154187+0000","last_clean":"2024-09-18T01:00:45.154187+0000","last_became_active":"2024-09-18T01:00:23.943957+0000","last_became_peered":"2024-09-18T01:00:23.943957+0000","last_unstale":"2024-09-18T01:00:45.154187+0000","last_undegraded":"2024-09-18T01:00:45.154187+0000","last_fullsized":"2024-09-18T01:00:45.154187+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:30.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027658140000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5114,"num_read_kb":26799,"num_write":585,"num_write_kb":11735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":9379374,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":482,"num_read":220595,"num_read_kb":1067292,"num_write":22261,"num_write_kb":688577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9848,"num_bytes_recovered":693846180,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":189480960,"data_stored":238352656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19538,"ondisk_log_size":19538,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47268,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324572,"statfs":{"total":96636764160,"available":96588361728,"internally_reserved":0,"allocated":2240512,"data_stored":1076641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47860,"kb_used_data":2780,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323980,"statfs":{"total":96636764160,"available":96587755520,"internally_reserved":0,"allocated":2846720,"data_stored":1672850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55604,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316236,"statfs":{"total":96636764160,"available":96579825664,"internally_reserved":0,"allocated":2248704,"data_stored":1082463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509486,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55600,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316240,"statfs":{"total":96636764160,"available":96579829760,"internally_reserved":0,"allocated":2248704,"data_stored":1082463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":64424509486,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52024,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319816,"statfs":{"total":96636764160,"available":96583491584,"internally_reserved":0,"allocated":2850816,"data_stored":1672869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":64424509485,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2285568,"data_stored":1100399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136798208,"data_stored":170151383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52674560,"data_stored":68201235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:00:50.525 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:00:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:00:50.822 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T01:00:50.838 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-18T01:00:50.173306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":482,"num_read":220711,"num_read_kb":1067392,"num_write":22422,"num_write_kb":690995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9854,"num_bytes_recovered":693846218,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19658,"ondisk_log_size":19658,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":309828,"kb_used_data":14376,"kb_used_omap":9,"kb_used_meta":295286,"kb_avail":565921212,"statfs":{"total":579820584960,"available":579503321088,"internally_reserved":0,"allocated":14721024,"data_stored":7687685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":302373542},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016185"},"pg_stats":[{"pgid":"3.1f","version":"186'621","reported_seq":4171,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572122+0000","last_change":"2024-09-18T01:00:44.155176+0000","last_active":"2024-09-18T01:00:45.572122+0000","last_peered":"2024-09-18T01:00:45.572122+0000","last_clean":"2024-09-18T01:00:45.572122+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T01:00:45.572122+0000","last_undegraded":"2024-09-18T01:00:45.572122+0000","last_fullsized":"2024-09-18T01:00:45.572122+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:49.598474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001427074,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6437,"num_read_kb":23165,"num_write":614,"num_write_kb":11006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.1e","version":"186'646","reported_seq":5444,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572219+0000","last_change":"2024-09-18T01:00:44.158389+0000","last_active":"2024-09-18T01:00:45.572219+0000","last_peered":"2024-09-18T01:00:45.572219+0000","last_clean":"2024-09-18T01:00:45.572219+0000","last_became_active":"2024-09-18T01:00:23.947401+0000","last_became_peered":"2024-09-18T01:00:23.947401+0000","last_unstale":"2024-09-18T01:00:45.572219+0000","last_undegraded":"2024-09-18T01:00:45.572219+0000","last_fullsized":"2024-09-18T01:00:45.572219+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:39:19.159376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041620060000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4822,"num_read_kb":36392,"num_write":703,"num_write_kb":11559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":30825045,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.1d","version":"184'800","reported_seq":12660,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.154432+0000","last_change":"2024-09-18T01:00:44.147606+0000","last_active":"2024-09-18T01:00:45.154432+0000","last_peered":"2024-09-18T01:00:45.154432+0000","last_clean":"2024-09-18T01:00:45.154432+0000","last_became_active":"2024-09-18T01:00:20.005875+0000","last_became_peered":"2024-09-18T01:00:20.005875+0000","last_unstale":"2024-09-18T01:00:45.154432+0000","last_undegraded":"2024-09-18T01:00:45.154432+0000","last_fullsized":"2024-09-18T01:00:45.154432+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:08.761605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034116569999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":13334,"num_read_kb":40113,"num_write":945,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":22094834,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.1c","version":"186'663","reported_seq":4532,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.709971+0000","last_change":"2024-09-18T01:00:44.151679+0000","last_active":"2024-09-18T01:00:45.709971+0000","last_peered":"2024-09-18T01:00:45.709971+0000","last_clean":"2024-09-18T01:00:45.709971+0000","last_became_active":"2024-09-18T00:59:17.918223+0000","last_became_peered":"2024-09-18T00:59:17.918223+0000","last_unstale":"2024-09-18T01:00:45.709971+0000","last_undegraded":"2024-09-18T01:00:45.709971+0000","last_fullsized":"2024-09-18T01:00:45.709971+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:30:03.649520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002343262,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5693,"num_read_kb":28138,"num_write":736,"num_write_kb":11405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":5428683,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.1b","version":"186'1093","reported_seq":10545,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156312+0000","last_change":"2024-09-18T01:00:44.149683+0000","last_active":"2024-09-18T01:00:45.156312+0000","last_peered":"2024-09-18T01:00:45.156312+0000","last_clean":"2024-09-18T01:00:45.156312+0000","last_became_active":"2024-09-18T01:00:22.941911+0000","last_became_peered":"2024-09-18T01:00:22.941911+0000","last_unstale":"2024-09-18T01:00:45.156312+0000","last_undegraded":"2024-09-18T01:00:45.156312+0000","last_fullsized":"2024-09-18T01:00:45.156312+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:31:18.554987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003279527,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12686,"num_read_kb":40787,"num_write":1375,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":495,"num_bytes_recovered":59340116,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.1a","version":"186'364","reported_seq":5092,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.709995+0000","last_change":"2024-09-18T01:00:44.151068+0000","last_active":"2024-09-18T01:00:45.709995+0000","last_peered":"2024-09-18T01:00:45.709995+0000","last_clean":"2024-09-18T01:00:45.709995+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T01:00:45.709995+0000","last_undegraded":"2024-09-18T01:00:45.709995+0000","last_fullsized":"2024-09-18T01:00:45.709995+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:24.750570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017298630000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5410,"num_read_kb":29581,"num_write":319,"num_write_kb":12768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.19","version":"186'701","reported_seq":30835,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.709738+0000","last_change":"2024-09-18T01:00:44.150671+0000","last_active":"2024-09-18T01:00:45.709738+0000","last_peered":"2024-09-18T01:00:45.709738+0000","last_clean":"2024-09-18T01:00:45.709738+0000","last_became_active":"2024-09-18T01:00:10.952126+0000","last_became_peered":"2024-09-18T01:00:10.952126+0000","last_unstale":"2024-09-18T01:00:45.709738+0000","last_undegraded":"2024-09-18T01:00:45.709738+0000","last_fullsized":"2024-09-18T01:00:45.709738+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:20:32.250982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001459178,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":30614,"num_read_kb":78260,"num_write":886,"num_write_kb":27079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":40291284,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.18","version":"186'656","reported_seq":3965,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572006+0000","last_change":"2024-09-18T01:00:44.157985+0000","last_active":"2024-09-18T01:00:45.572006+0000","last_peered":"2024-09-18T01:00:45.572006+0000","last_clean":"2024-09-18T01:00:45.572006+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T01:00:45.572006+0000","last_undegraded":"2024-09-18T01:00:45.572006+0000","last_fullsized":"2024-09-18T01:00:45.572006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:27.075227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039694869999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4869,"num_read_kb":13814,"num_write":877,"num_write_kb":42194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.17","version":"185'676","reported_seq":7303,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572211+0000","last_change":"2024-09-18T01:00:44.156705+0000","last_active":"2024-09-18T01:00:45.572211+0000","last_peered":"2024-09-18T01:00:45.572211+0000","last_clean":"2024-09-18T01:00:45.572211+0000","last_became_active":"2024-09-18T01:00:20.017711+0000","last_became_peered":"2024-09-18T01:00:20.017711+0000","last_unstale":"2024-09-18T01:00:45.572211+0000","last_undegraded":"2024-09-18T01:00:45.572211+0000","last_fullsized":"2024-09-18T01:00:45.572211+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:49.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028584439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7284,"num_read_kb":46982,"num_write":773,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":13346337,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.16","version":"186'641","reported_seq":8661,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.571866+0000","last_change":"2024-09-18T01:00:44.155480+0000","last_active":"2024-09-18T01:00:45.571866+0000","last_peered":"2024-09-18T01:00:45.571866+0000","last_clean":"2024-09-18T01:00:45.571866+0000","last_became_active":"2024-09-18T01:00:31.070442+0000","last_became_peered":"2024-09-18T01:00:31.070442+0000","last_unstale":"2024-09-18T01:00:45.571866+0000","last_undegraded":"2024-09-18T01:00:45.571866+0000","last_fullsized":"2024-09-18T01:00:45.571866+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:43:37.754762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887849999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8507,"num_read_kb":37252,"num_write":699,"num_write_kb":22150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":37601010,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.15","version":"184'519","reported_seq":2989,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.261870+0000","last_change":"2024-09-18T01:00:44.262860+0000","last_active":"2024-09-18T01:00:44.261870+0000","last_peered":"2024-09-18T01:00:44.261870+0000","last_clean":"2024-09-18T01:00:44.261870+0000","last_became_active":"2024-09-18T01:00:21.007605+0000","last_became_peered":"2024-09-18T01:00:21.007605+0000","last_unstale":"2024-09-18T01:00:44.261870+0000","last_undegraded":"2024-09-18T01:00:44.261870+0000","last_fullsized":"2024-09-18T01:00:44.261870+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:20:41.796582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035212300000000002,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2322,"num_read_kb":15300,"num_write":809,"num_write_kb":48070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":22291504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.14","version":"181'371","reported_seq":2973,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156375+0000","last_change":"2024-09-18T01:00:44.147456+0000","last_active":"2024-09-18T01:00:45.156375+0000","last_peered":"2024-09-18T01:00:45.156375+0000","last_clean":"2024-09-18T01:00:45.156375+0000","last_became_active":"2024-09-18T01:00:31.066969+0000","last_became_peered":"2024-09-18T01:00:31.066969+0000","last_unstale":"2024-09-18T01:00:45.156375+0000","last_undegraded":"2024-09-18T01:00:45.156375+0000","last_fullsized":"2024-09-18T01:00:45.156375+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:25:29.555120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013361650000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2452,"num_read_kb":3816,"num_write":339,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":469,"num_bytes_recovered":3474495,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.154075+0000","last_change":"2024-09-18T01:00:15.995790+0000","last_active":"2024-09-18T01:00:45.154075+0000","last_peered":"2024-09-18T01:00:45.154075+0000","last_clean":"2024-09-18T01:00:45.154075+0000","last_became_active":"2024-09-18T01:00:15.995425+0000","last_became_peered":"2024-09-18T01:00:15.995425+0000","last_unstale":"2024-09-18T01:00:45.154075+0000","last_undegraded":"2024-09-18T01:00:45.154075+0000","last_fullsized":"2024-09-18T01:00:45.154075+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:49.875513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"186'777","reported_seq":4829,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156357+0000","last_change":"2024-09-18T01:00:44.148713+0000","last_active":"2024-09-18T01:00:45.156357+0000","last_peered":"2024-09-18T01:00:45.156357+0000","last_clean":"2024-09-18T01:00:45.156357+0000","last_became_active":"2024-09-18T01:00:11.936218+0000","last_became_peered":"2024-09-18T01:00:11.936218+0000","last_unstale":"2024-09-18T01:00:45.156357+0000","last_undegraded":"2024-09-18T01:00:45.156357+0000","last_fullsized":"2024-09-18T01:00:45.156357+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:24.939297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002351681,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5488,"num_read_kb":22949,"num_write":1002,"num_write_kb":4650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":2805113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":353,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156369+0000","last_change":"2024-09-18T00:59:15.073941+0000","last_active":"2024-09-18T01:00:45.156369+0000","last_peered":"2024-09-18T01:00:45.156369+0000","last_clean":"2024-09-18T01:00:45.156369+0000","last_became_active":"2024-09-18T00:59:15.072986+0000","last_became_peered":"2024-09-18T00:59:15.072986+0000","last_unstale":"2024-09-18T01:00:45.156369+0000","last_undegraded":"2024-09-18T01:00:45.156369+0000","last_fullsized":"2024-09-18T01:00:45.156369+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:16.671308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"185'506","reported_seq":8829,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156623+0000","last_change":"2024-09-18T01:00:44.150561+0000","last_active":"2024-09-18T01:00:45.156623+0000","last_peered":"2024-09-18T01:00:45.156623+0000","last_clean":"2024-09-18T01:00:45.156623+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T01:00:45.156623+0000","last_undegraded":"2024-09-18T01:00:45.156623+0000","last_fullsized":"2024-09-18T01:00:45.156623+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:44.745709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036716330000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8918,"num_read_kb":44936,"num_write":410,"num_write_kb":13864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.154571+0000","last_change":"2024-09-18T01:00:15.991240+0000","last_active":"2024-09-18T01:00:45.154571+0000","last_peered":"2024-09-18T01:00:45.154571+0000","last_clean":"2024-09-18T01:00:45.154571+0000","last_became_active":"2024-09-18T01:00:15.990930+0000","last_became_peered":"2024-09-18T01:00:15.990930+0000","last_unstale":"2024-09-18T01:00:45.154571+0000","last_undegraded":"2024-09-18T01:00:45.154571+0000","last_fullsized":"2024-09-18T01:00:45.154571+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:55.616429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"185'348","reported_seq":4436,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.260079+0000","last_change":"2024-09-18T01:00:44.260854+0000","last_active":"2024-09-18T01:00:44.260079+0000","last_peered":"2024-09-18T01:00:44.260079+0000","last_clean":"2024-09-18T01:00:44.260079+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T01:00:44.260079+0000","last_undegraded":"2024-09-18T01:00:44.260079+0000","last_fullsized":"2024-09-18T01:00:44.260079+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:09.012851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019390989999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3644,"num_read_kb":27224,"num_write":289,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":347,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572005+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T01:00:45.572005+0000","last_peered":"2024-09-18T01:00:45.572005+0000","last_clean":"2024-09-18T01:00:45.572005+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T01:00:45.572005+0000","last_undegraded":"2024-09-18T01:00:45.572005+0000","last_fullsized":"2024-09-18T01:00:45.572005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"185'580","reported_seq":4466,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.260943+0000","last_change":"2024-09-18T01:00:44.261780+0000","last_active":"2024-09-18T01:00:44.260943+0000","last_peered":"2024-09-18T01:00:44.260943+0000","last_clean":"2024-09-18T01:00:44.260943+0000","last_became_active":"2024-09-18T00:58:58.963883+0000","last_became_peered":"2024-09-18T00:58:58.963883+0000","last_unstale":"2024-09-18T01:00:44.260943+0000","last_undegraded":"2024-09-18T01:00:44.260943+0000","last_fullsized":"2024-09-18T01:00:44.260943+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:34:48.165282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027378979999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4830,"num_read_kb":33562,"num_write":800,"num_write_kb":24925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.3","version":"186'946","reported_seq":6113,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156120+0000","last_change":"2024-09-18T01:00:44.148549+0000","last_active":"2024-09-18T01:00:45.156120+0000","last_peered":"2024-09-18T01:00:45.156120+0000","last_clean":"2024-09-18T01:00:45.156120+0000","last_became_active":"2024-09-18T01:00:21.926403+0000","last_became_peered":"2024-09-18T01:00:21.926403+0000","last_unstale":"2024-09-18T01:00:45.156120+0000","last_undegraded":"2024-09-18T01:00:45.156120+0000","last_fullsized":"2024-09-18T01:00:45.156120+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:09:46.795184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022401439999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7645,"num_read_kb":19179,"num_write":1076,"num_write_kb":33944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":99850853,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.2","version":"23'2","reported_seq":204,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.259234+0000","last_change":"2024-09-18T01:00:28.086666+0000","last_active":"2024-09-18T01:00:44.259234+0000","last_peered":"2024-09-18T01:00:44.259234+0000","last_clean":"2024-09-18T01:00:44.259234+0000","last_became_active":"2024-09-18T01:00:28.078264+0000","last_became_peered":"2024-09-18T01:00:28.078264+0000","last_unstale":"2024-09-18T01:00:44.259234+0000","last_undegraded":"2024-09-18T01:00:44.259234+0000","last_fullsized":"2024-09-18T01:00:44.259234+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:15.282919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"185'657","reported_seq":8330,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.571628+0000","last_change":"2024-09-18T01:00:44.156905+0000","last_active":"2024-09-18T01:00:45.571628+0000","last_peered":"2024-09-18T01:00:45.571628+0000","last_clean":"2024-09-18T01:00:45.571628+0000","last_became_active":"2024-09-18T01:00:08.951992+0000","last_became_peered":"2024-09-18T01:00:08.951992+0000","last_unstale":"2024-09-18T01:00:45.571628+0000","last_undegraded":"2024-09-18T01:00:45.571628+0000","last_fullsized":"2024-09-18T01:00:45.571628+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:59.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030555019999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7663,"num_read_kb":29924,"num_write":748,"num_write_kb":19563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":15190015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":95,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-18T01:00:47.938432+0000","last_change":"2024-09-18T01:00:04.976725+0000","last_active":"2024-09-18T01:00:47.938432+0000","last_peered":"2024-09-18T01:00:47.938432+0000","last_clean":"2024-09-18T01:00:47.938432+0000","last_became_active":"2024-09-18T01:00:04.976360+0000","last_became_peered":"2024-09-18T01:00:04.976360+0000","last_unstale":"2024-09-18T01:00:47.938432+0000","last_undegraded":"2024-09-18T01:00:47.938432+0000","last_fullsized":"2024-09-18T01:00:47.938432+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:11.159126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"184'548","reported_seq":9364,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.259969+0000","last_change":"2024-09-18T01:00:44.261168+0000","last_active":"2024-09-18T01:00:44.259969+0000","last_peered":"2024-09-18T01:00:44.259969+0000","last_clean":"2024-09-18T01:00:44.259969+0000","last_became_active":"2024-09-18T01:00:31.069677+0000","last_became_peered":"2024-09-18T01:00:31.069677+0000","last_unstale":"2024-09-18T01:00:44.259969+0000","last_undegraded":"2024-09-18T01:00:44.259969+0000","last_fullsized":"2024-09-18T01:00:44.259969+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:38:57.075708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726180000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8919,"num_read_kb":48544,"num_write":559,"num_write_kb":18757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":794,"num_bytes_recovered":19034719,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.153462+0000","last_change":"2024-09-18T01:00:15.994517+0000","last_active":"2024-09-18T01:00:45.153462+0000","last_peered":"2024-09-18T01:00:45.153462+0000","last_clean":"2024-09-18T01:00:45.153462+0000","last_became_active":"2024-09-18T01:00:15.994159+0000","last_became_peered":"2024-09-18T01:00:15.994159+0000","last_unstale":"2024-09-18T01:00:45.153462+0000","last_undegraded":"2024-09-18T01:00:45.153462+0000","last_fullsized":"2024-09-18T01:00:45.153462+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:06.775713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"186'722","reported_seq":5041,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156024+0000","last_change":"2024-09-18T01:00:44.147483+0000","last_active":"2024-09-18T01:00:45.156024+0000","last_peered":"2024-09-18T01:00:45.156024+0000","last_clean":"2024-09-18T01:00:45.156024+0000","last_became_active":"2024-09-18T00:59:18.918527+0000","last_became_peered":"2024-09-18T00:59:18.918527+0000","last_unstale":"2024-09-18T01:00:45.156024+0000","last_undegraded":"2024-09-18T01:00:45.156024+0000","last_fullsized":"2024-09-18T01:00:45.156024+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:31.958835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00139405,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5240,"num_read_kb":16439,"num_write":1146,"num_write_kb":52135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":12961092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"2.3","version":"21'1","reported_seq":81,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.153959+0000","last_change":"2024-09-18T01:00:28.080162+0000","last_active":"2024-09-18T01:00:45.153959+0000","last_peered":"2024-09-18T01:00:45.153959+0000","last_clean":"2024-09-18T01:00:45.153959+0000","last_became_active":"2024-09-18T01:00:28.075757+0000","last_became_peered":"2024-09-18T01:00:28.075757+0000","last_unstale":"2024-09-18T01:00:45.153959+0000","last_undegraded":"2024-09-18T01:00:45.153959+0000","last_fullsized":"2024-09-18T01:00:45.153959+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:29.327404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":499,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.709650+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T01:00:45.709650+0000","last_peered":"2024-09-18T01:00:45.709650+0000","last_clean":"2024-09-18T01:00:45.709650+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T01:00:45.709650+0000","last_undegraded":"2024-09-18T01:00:45.709650+0000","last_fullsized":"2024-09-18T01:00:45.709650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"184'448","reported_seq":5825,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.156006+0000","last_change":"2024-09-18T01:00:44.149901+0000","last_active":"2024-09-18T01:00:45.156006+0000","last_peered":"2024-09-18T01:00:45.156006+0000","last_clean":"2024-09-18T01:00:45.156006+0000","last_became_active":"2024-09-18T01:00:08.949384+0000","last_became_peered":"2024-09-18T01:00:08.949384+0000","last_unstale":"2024-09-18T01:00:45.156006+0000","last_undegraded":"2024-09-18T01:00:45.156006+0000","last_fullsized":"2024-09-18T01:00:45.156006+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:20.064612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033952990000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5722,"num_read_kb":51790,"num_write":458,"num_write_kb":39546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":32435091,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.9","version":"185'408","reported_seq":3728,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.153549+0000","last_change":"2024-09-18T01:00:44.145255+0000","last_active":"2024-09-18T01:00:45.153549+0000","last_peered":"2024-09-18T01:00:45.153549+0000","last_clean":"2024-09-18T01:00:45.153549+0000","last_became_active":"2024-09-18T01:00:18.997710+0000","last_became_peered":"2024-09-18T01:00:18.997710+0000","last_unstale":"2024-09-18T01:00:45.153549+0000","last_undegraded":"2024-09-18T01:00:45.153549+0000","last_fullsized":"2024-09-18T01:00:45.153549+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:36.454597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016382809999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3625,"num_read_kb":11553,"num_write":342,"num_write_kb":23969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":27148173,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.a","version":"185'607","reported_seq":5574,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.153657+0000","last_change":"2024-09-18T01:00:44.146011+0000","last_active":"2024-09-18T01:00:45.153657+0000","last_peered":"2024-09-18T01:00:45.153657+0000","last_clean":"2024-09-18T01:00:45.153657+0000","last_became_active":"2024-09-18T01:00:22.938344+0000","last_became_peered":"2024-09-18T01:00:22.938344+0000","last_unstale":"2024-09-18T01:00:45.153657+0000","last_undegraded":"2024-09-18T01:00:45.153657+0000","last_fullsized":"2024-09-18T01:00:45.153657+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:59:54.570595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021924589999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4754,"num_read_kb":46819,"num_write":596,"num_write_kb":17196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":6639875,"num_keys_recovered":85,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.b","version":"185'515","reported_seq":5867,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-18T01:00:47.938035+0000","last_change":"2024-09-18T01:00:44.262988+0000","last_active":"2024-09-18T01:00:47.938035+0000","last_peered":"2024-09-18T01:00:47.938035+0000","last_clean":"2024-09-18T01:00:47.938035+0000","last_became_active":"2024-09-18T01:00:10.954910+0000","last_became_peered":"2024-09-18T01:00:10.954910+0000","last_unstale":"2024-09-18T01:00:47.938035+0000","last_undegraded":"2024-09-18T01:00:47.938035+0000","last_fullsized":"2024-09-18T01:00:47.938035+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:45.069175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042875949999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5466,"num_read_kb":66445,"num_write":496,"num_write_kb":15031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15399944,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"4"}]},{"pgid":"3.c","version":"185'566","reported_seq":7089,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-18T01:00:47.938077+0000","last_change":"2024-09-18T01:00:44.261560+0000","last_active":"2024-09-18T01:00:47.938077+0000","last_peered":"2024-09-18T01:00:47.938077+0000","last_clean":"2024-09-18T01:00:47.938077+0000","last_became_active":"2024-09-18T01:00:19.999750+0000","last_became_peered":"2024-09-18T01:00:19.999750+0000","last_unstale":"2024-09-18T01:00:47.938077+0000","last_undegraded":"2024-09-18T01:00:47.938077+0000","last_fullsized":"2024-09-18T01:00:47.938077+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:13:23.055801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030770020000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6401,"num_read_kb":47394,"num_write":633,"num_write_kb":22377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":26783458,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"4"}]},{"pgid":"3.d","version":"184'976","reported_seq":6750,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.154283+0000","last_change":"2024-09-18T01:00:44.146438+0000","last_active":"2024-09-18T01:00:45.154283+0000","last_peered":"2024-09-18T01:00:45.154283+0000","last_clean":"2024-09-18T01:00:45.154283+0000","last_became_active":"2024-09-18T01:00:21.923279+0000","last_became_peered":"2024-09-18T01:00:21.923279+0000","last_unstale":"2024-09-18T01:00:45.154283+0000","last_undegraded":"2024-09-18T01:00:45.154283+0000","last_fullsized":"2024-09-18T01:00:45.154283+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:23.093467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026100659999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8636,"num_read_kb":40955,"num_write":1143,"num_write_kb":30743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":80305919,"num_keys_recovered":163,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.e","version":"183'713","reported_seq":4734,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.572023+0000","last_change":"2024-09-18T01:00:44.157454+0000","last_active":"2024-09-18T01:00:45.572023+0000","last_peered":"2024-09-18T01:00:45.572023+0000","last_clean":"2024-09-18T01:00:45.572023+0000","last_became_active":"2024-09-18T01:00:21.008903+0000","last_became_peered":"2024-09-18T01:00:21.008903+0000","last_unstale":"2024-09-18T01:00:45.572023+0000","last_undegraded":"2024-09-18T01:00:45.572023+0000","last_fullsized":"2024-09-18T01:00:45.572023+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:53.068038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034902890000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4741,"num_read_kb":29100,"num_write":948,"num_write_kb":26466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":41587165,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.f","version":"184'563","reported_seq":4323,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.710171+0000","last_change":"2024-09-18T01:00:44.152013+0000","last_active":"2024-09-18T01:00:45.710171+0000","last_peered":"2024-09-18T01:00:45.710171+0000","last_clean":"2024-09-18T01:00:45.710171+0000","last_became_active":"2024-09-18T00:59:17.917905+0000","last_became_peered":"2024-09-18T00:59:17.917905+0000","last_unstale":"2024-09-18T01:00:45.710171+0000","last_undegraded":"2024-09-18T01:00:45.710171+0000","last_fullsized":"2024-09-18T01:00:45.710171+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:51:06.807009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024190230000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5311,"num_read_kb":28191,"num_write":598,"num_write_kb":9287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":6320640,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.10","version":"184'456","reported_seq":5250,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.709630+0000","last_change":"2024-09-18T01:00:44.148986+0000","last_active":"2024-09-18T01:00:45.709630+0000","last_peered":"2024-09-18T01:00:45.709630+0000","last_clean":"2024-09-18T01:00:45.709630+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T01:00:45.709630+0000","last_undegraded":"2024-09-18T01:00:45.709630+0000","last_fullsized":"2024-09-18T01:00:45.709630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:48:20.609462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001443239,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":51584,"num_write":357,"num_write_kb":26802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.11","version":"183'292","reported_seq":3191,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.153805+0000","last_change":"2024-09-18T01:00:44.147261+0000","last_active":"2024-09-18T01:00:45.153805+0000","last_peered":"2024-09-18T01:00:45.153805+0000","last_clean":"2024-09-18T01:00:45.153805+0000","last_became_active":"2024-09-18T01:00:19.002365+0000","last_became_peered":"2024-09-18T01:00:19.002365+0000","last_unstale":"2024-09-18T01:00:45.153805+0000","last_undegraded":"2024-09-18T01:00:45.153805+0000","last_fullsized":"2024-09-18T01:00:45.153805+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:37:21.382231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033253200000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1897,"num_read_kb":13175,"num_write":246,"num_write_kb":14260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":15824068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.12","version":"184'638","reported_seq":6854,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-18T01:00:44.261237+0000","last_change":"2024-09-18T01:00:44.261997+0000","last_active":"2024-09-18T01:00:44.261237+0000","last_peered":"2024-09-18T01:00:44.261237+0000","last_clean":"2024-09-18T01:00:44.261237+0000","last_became_active":"2024-09-18T01:00:21.926814+0000","last_became_peered":"2024-09-18T01:00:21.926814+0000","last_unstale":"2024-09-18T01:00:44.261237+0000","last_undegraded":"2024-09-18T01:00:44.261237+0000","last_fullsized":"2024-09-18T01:00:44.261237+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:04.145432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002915932,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6954,"num_read_kb":17130,"num_write":754,"num_write_kb":11285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":32804395,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"9"}]},{"pgid":"3.13","version":"185'521","reported_seq":4574,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-18T01:00:45.154187+0000","last_change":"2024-09-18T01:00:44.148400+0000","last_active":"2024-09-18T01:00:45.154187+0000","last_peered":"2024-09-18T01:00:45.154187+0000","last_clean":"2024-09-18T01:00:45.154187+0000","last_became_active":"2024-09-18T01:00:23.943957+0000","last_became_peered":"2024-09-18T01:00:23.943957+0000","last_unstale":"2024-09-18T01:00:45.154187+0000","last_undegraded":"2024-09-18T01:00:45.154187+0000","last_fullsized":"2024-09-18T01:00:45.154187+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:30.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027658140000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5114,"num_read_kb":26799,"num_write":585,"num_write_kb":11735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":9379374,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":482,"num_read":220595,"num_read_kb":1067292,"num_write":22261,"num_write_kb":688577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9848,"num_bytes_recovered":693846180,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":189480960,"data_stored":238352656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19538,"ondisk_log_size":19538,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47268,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324572,"statfs":{"total":96636764160,"available":96588361728,"internally_reserved":0,"allocated":2240512,"data_stored":1076641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47860,"kb_used_data":2780,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323980,"statfs":{"total":96636764160,"available":96587755520,"internally_reserved":0,"allocated":2846720,"data_stored":1672850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55604,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316236,"statfs":{"total":96636764160,"available":96579825664,"internally_reserved":0,"allocated":2248704,"data_stored":1082463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509486,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55600,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316240,"statfs":{"total":96636764160,"available":96579829760,"internally_reserved":0,"allocated":2248704,"data_stored":1082463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":64424509486,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52024,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319816,"statfs":{"total":96636764160,"available":96583491584,"internally_reserved":0,"allocated":2850816,"data_stored":1672869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":64424509485,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2285568,"data_stored":1100399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136798208,"data_stored":170151383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52674560,"data_stored":68201235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:00:50.840 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:00:50.841 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:00:50.841 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T01:00:50.841 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.952385 2024-09-18T01:00:50.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0.9523850408074744 2024-09-18T01:00:51.175 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.7 primary-affinity to 0.952385 (8624152) 2024-09-18T01:00:56.200 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-18T01:00:56.200 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-18T01:00:56.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-18T01:00:56.526 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-18T01:00:56.524+0000 7fdb35eeb640 -1 mon.a@0(leader).osd e196 definitely_dead 0 2024-09-18T01:00:57.228 INFO:teuthology.orchestra.run.smithi042.stderr:marked in osd.2. 2024-09-18T01:00:57.255 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-18T01:00:57.255 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-18T01:00:57.255 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-18T01:00:57.255 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-18T01:00:57.569 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-18T01:00:57.564+0000 7fdb35eeb640 -1 mon.a@0(leader).osd e197 definitely_dead 0 2024-09-18T01:00:58.215 INFO:teuthology.orchestra.run.smithi042.stderr:marked in osd.0. 2024-09-18T01:00:58.253 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-18T01:00:58.253 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-18T01:00:58.387 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a: {} 2024-09-18T01:00:58.387 INFO:teuthology.orchestra.run.smithi042.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:00:58.400 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b: {} 2024-09-18T01:00:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:00:58.410 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c: {} 2024-09-18T01:00:58.410 INFO:teuthology.orchestra.run.smithi042.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:00:58.427 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:00:59.237 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:00:59.286 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-18T01:01:00.228 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-18T01:01:00.294 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:01:01.237 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:01:01.266 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-18T01:01:01.936 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-18T01:01:01.973 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:01:02.943 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:01:02.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-18T01:01:03.952 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-18T01:01:03.979 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:01:05.297 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:01:05.322 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-18T01:01:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-18T01:01:06.325 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:01:07.309 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:01:07.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-18T01:01:08.318 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-18T01:01:08.348 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T01:01:09.328 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T01:01:09.356 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-18T01:01:10.339 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-18T01:01:10.368 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T01:01:11.348 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T01:01:11.372 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-18T01:01:12.357 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-18T01:01:12.381 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T01:01:13.360 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T01:01:13.386 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-18T01:01:14.382 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-18T01:01:14.410 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-18T01:01:14.411 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-18T01:01:14.411 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-18T01:01:14.411 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-18T01:01:14.411 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T01:01:14.411 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:01:14.721 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:01:14.722 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":214,"fsid":"82a96d76-ba83-49e2-a985-84604bd599ba","created":"2024-09-18T00:57:07.138732+0000","modified":"2024-09-18T01:01:14.367493+0000","last_up_change":"2024-09-18T00:57:20.258605+0000","last_in_change":"2024-09-18T01:00:58.211018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:57:21.962521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T00:57:25.721733+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T00:57:35.599413+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":186,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f359ce5-6f6e-4161-8554-8ac028347274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6801","nonce":199763443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6803","nonce":199763443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6807","nonce":199763443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6805","nonce":199763443}]},"public_addr":"172.21.15.42:6801/199763443","cluster_addr":"172.21.15.42:6803/199763443","heartbeat_back_addr":"172.21.15.42:6807/199763443","heartbeat_front_addr":"172.21.15.42:6805/199763443","state":["exists","up"]},{"osd":1,"uuid":"48687611-1556-4b6d-86af-7872494a91d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6809","nonce":345149800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6811","nonce":345149800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6815","nonce":345149800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6813","nonce":345149800}]},"public_addr":"172.21.15.42:6809/345149800","cluster_addr":"172.21.15.42:6811/345149800","heartbeat_back_addr":"172.21.15.42:6815/345149800","heartbeat_front_addr":"172.21.15.42:6813/345149800","state":["exists","up"]},{"osd":2,"uuid":"9e6b1f90-ce9b-46d4-92fe-d457f7e415d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6825","nonce":431239931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6827","nonce":431239931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6831","nonce":431239931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6829","nonce":431239931}]},"public_addr":"172.21.15.42:6825/431239931","cluster_addr":"172.21.15.42:6827/431239931","heartbeat_back_addr":"172.21.15.42:6831/431239931","heartbeat_front_addr":"172.21.15.42:6829/431239931","state":["exists","up"]},{"osd":3,"uuid":"dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6817","nonce":1775474180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6819","nonce":1775474180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6823","nonce":1775474180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6821","nonce":1775474180}]},"public_addr":"172.21.15.42:6817/1775474180","cluster_addr":"172.21.15.42:6819/1775474180","heartbeat_back_addr":"172.21.15.42:6823/1775474180","heartbeat_front_addr":"172.21.15.42:6821/1775474180","state":["exists","up"]},{"osd":4,"uuid":"802ccbeb-7351-4b38-832a-695117c7e493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6809","nonce":4198401207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6811","nonce":4198401207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6815","nonce":4198401207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6813","nonce":4198401207}]},"public_addr":"172.21.15.195:6809/4198401207","cluster_addr":"172.21.15.195:6811/4198401207","heartbeat_back_addr":"172.21.15.195:6815/4198401207","heartbeat_front_addr":"172.21.15.195:6813/4198401207","state":["exists","up"]},{"osd":5,"uuid":"d958b3da-efa1-4d86-b81d-e78874441437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6825","nonce":2599825823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6827","nonce":2599825823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6831","nonce":2599825823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6829","nonce":2599825823}]},"public_addr":"172.21.15.195:6825/2599825823","cluster_addr":"172.21.15.195:6827/2599825823","heartbeat_back_addr":"172.21.15.195:6831/2599825823","heartbeat_front_addr":"172.21.15.195:6829/2599825823","state":["exists","up"]},{"osd":6,"uuid":"51b3cea8-c686-4fd3-b3f4-ba936cf26b64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6801","nonce":3550656451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6803","nonce":3550656451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6807","nonce":3550656451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6805","nonce":3550656451}]},"public_addr":"172.21.15.195:6801/3550656451","cluster_addr":"172.21.15.195:6803/3550656451","heartbeat_back_addr":"172.21.15.195:6807/3550656451","heartbeat_front_addr":"172.21.15.195:6805/3550656451","state":["exists","up"]},{"osd":7,"uuid":"b02ead1c-837a-4b07-9ad1-4e903ed5bc4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6817","nonce":2993352359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6819","nonce":2993352359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6823","nonce":2993352359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6821","nonce":2993352359}]},"public_addr":"172.21.15.195:6817/2993352359","cluster_addr":"172.21.15.195:6819/2993352359","heartbeat_back_addr":"172.21.15.195:6823/2993352359","heartbeat_front_addr":"172.21.15.195:6821/2993352359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.896477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.941756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.560180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.077394+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.996264+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.511686+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.411051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.888092+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:01:14.739 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T01:01:14.740 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:01:15.028 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:01:15.029 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":214,"fsid":"82a96d76-ba83-49e2-a985-84604bd599ba","created":"2024-09-18T00:57:07.138732+0000","modified":"2024-09-18T01:01:14.367493+0000","last_up_change":"2024-09-18T00:57:20.258605+0000","last_in_change":"2024-09-18T01:00:58.211018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:57:21.962521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T00:57:25.721733+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T00:57:35.599413+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":186,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f359ce5-6f6e-4161-8554-8ac028347274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6801","nonce":199763443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6803","nonce":199763443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6807","nonce":199763443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6805","nonce":199763443}]},"public_addr":"172.21.15.42:6801/199763443","cluster_addr":"172.21.15.42:6803/199763443","heartbeat_back_addr":"172.21.15.42:6807/199763443","heartbeat_front_addr":"172.21.15.42:6805/199763443","state":["exists","up"]},{"osd":1,"uuid":"48687611-1556-4b6d-86af-7872494a91d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6809","nonce":345149800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6811","nonce":345149800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6815","nonce":345149800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6813","nonce":345149800}]},"public_addr":"172.21.15.42:6809/345149800","cluster_addr":"172.21.15.42:6811/345149800","heartbeat_back_addr":"172.21.15.42:6815/345149800","heartbeat_front_addr":"172.21.15.42:6813/345149800","state":["exists","up"]},{"osd":2,"uuid":"9e6b1f90-ce9b-46d4-92fe-d457f7e415d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6825","nonce":431239931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6827","nonce":431239931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6831","nonce":431239931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6829","nonce":431239931}]},"public_addr":"172.21.15.42:6825/431239931","cluster_addr":"172.21.15.42:6827/431239931","heartbeat_back_addr":"172.21.15.42:6831/431239931","heartbeat_front_addr":"172.21.15.42:6829/431239931","state":["exists","up"]},{"osd":3,"uuid":"dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6817","nonce":1775474180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6819","nonce":1775474180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6823","nonce":1775474180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6821","nonce":1775474180}]},"public_addr":"172.21.15.42:6817/1775474180","cluster_addr":"172.21.15.42:6819/1775474180","heartbeat_back_addr":"172.21.15.42:6823/1775474180","heartbeat_front_addr":"172.21.15.42:6821/1775474180","state":["exists","up"]},{"osd":4,"uuid":"802ccbeb-7351-4b38-832a-695117c7e493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6809","nonce":4198401207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6811","nonce":4198401207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6815","nonce":4198401207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6813","nonce":4198401207}]},"public_addr":"172.21.15.195:6809/4198401207","cluster_addr":"172.21.15.195:6811/4198401207","heartbeat_back_addr":"172.21.15.195:6815/4198401207","heartbeat_front_addr":"172.21.15.195:6813/4198401207","state":["exists","up"]},{"osd":5,"uuid":"d958b3da-efa1-4d86-b81d-e78874441437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6825","nonce":2599825823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6827","nonce":2599825823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6831","nonce":2599825823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6829","nonce":2599825823}]},"public_addr":"172.21.15.195:6825/2599825823","cluster_addr":"172.21.15.195:6827/2599825823","heartbeat_back_addr":"172.21.15.195:6831/2599825823","heartbeat_front_addr":"172.21.15.195:6829/2599825823","state":["exists","up"]},{"osd":6,"uuid":"51b3cea8-c686-4fd3-b3f4-ba936cf26b64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6801","nonce":3550656451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6803","nonce":3550656451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6807","nonce":3550656451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6805","nonce":3550656451}]},"public_addr":"172.21.15.195:6801/3550656451","cluster_addr":"172.21.15.195:6803/3550656451","heartbeat_back_addr":"172.21.15.195:6807/3550656451","heartbeat_front_addr":"172.21.15.195:6805/3550656451","state":["exists","up"]},{"osd":7,"uuid":"b02ead1c-837a-4b07-9ad1-4e903ed5bc4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6817","nonce":2993352359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6819","nonce":2993352359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6823","nonce":2993352359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6821","nonce":2993352359}]},"public_addr":"172.21.15.195:6817/2993352359","cluster_addr":"172.21.15.195:6819/2993352359","heartbeat_back_addr":"172.21.15.195:6823/2993352359","heartbeat_front_addr":"172.21.15.195:6821/2993352359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.896477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.941756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.560180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.077394+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.996264+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.511686+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.411051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.888092+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:01:15.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T01:01:15.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T01:01:15.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T01:01:15.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T01:01:15.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T01:01:15.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T01:01:15.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T01:01:15.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T01:01:15.317 INFO:teuthology.orchestra.run.smithi042.stdout:64424509490 2024-09-18T01:01:15.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T01:01:15.319 INFO:teuthology.orchestra.run.smithi042.stdout:64424509492 2024-09-18T01:01:15.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T01:01:15.325 INFO:teuthology.orchestra.run.smithi042.stdout:64424509492 2024-09-18T01:01:15.325 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T01:01:15.329 INFO:teuthology.orchestra.run.smithi042.stdout:55834574899 2024-09-18T01:01:15.329 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T01:01:15.331 INFO:teuthology.orchestra.run.smithi042.stdout:64424509490 2024-09-18T01:01:15.331 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T01:01:15.332 INFO:teuthology.orchestra.run.smithi042.stdout:64424509492 2024-09-18T01:01:15.333 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T01:01:15.337 INFO:teuthology.orchestra.run.smithi042.stdout:64424509491 2024-09-18T01:01:15.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T01:01:15.339 INFO:teuthology.orchestra.run.smithi042.stdout:64424509491 2024-09-18T01:01:15.340 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T01:01:15.826 INFO:teuthology.orchestra.run.smithi042.stdout:64424509491 2024-09-18T01:01:15.828 INFO:teuthology.orchestra.run.smithi042.stdout:64424509489 2024-09-18T01:01:15.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509491 for osd.4 2024-09-18T01:01:15.853 INFO:teuthology.orchestra.run.smithi042.stdout:64424509489 2024-09-18T01:01:15.854 INFO:teuthology.orchestra.run.smithi042.stdout:64424509491 2024-09-18T01:01:15.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509490 got 64424509489 for osd.5 2024-09-18T01:01:15.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509490 got 64424509489 for osd.6 2024-09-18T01:01:15.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509491 for osd.2 2024-09-18T01:01:15.885 INFO:teuthology.orchestra.run.smithi042.stdout:55834574898 2024-09-18T01:01:15.894 INFO:teuthology.orchestra.run.smithi042.stdout:64424509491 2024-09-18T01:01:15.906 INFO:teuthology.orchestra.run.smithi042.stdout:64424509490 2024-09-18T01:01:15.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.7 2024-09-18T01:01:15.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509491 for osd.3 2024-09-18T01:01:15.917 INFO:teuthology.orchestra.run.smithi042.stdout:64424509490 2024-09-18T01:01:15.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509490 for osd.0 2024-09-18T01:01:15.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509490 for osd.1 2024-09-18T01:01:16.852 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T01:01:16.860 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T01:01:16.883 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T01:01:16.884 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T01:01:16.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T01:01:16.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T01:01:16.926 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T01:01:16.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T01:01:17.377 INFO:teuthology.orchestra.run.smithi042.stdout:64424509492 2024-09-18T01:01:17.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509492 for osd.4 2024-09-18T01:01:17.402 DEBUG:teuthology.parallel:result is None 2024-09-18T01:01:17.416 INFO:teuthology.orchestra.run.smithi042.stdout:64424509491 2024-09-18T01:01:17.421 INFO:teuthology.orchestra.run.smithi042.stdout:64424509493 2024-09-18T01:01:17.437 INFO:teuthology.orchestra.run.smithi042.stdout:64424509490 2024-09-18T01:01:17.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509490 got 64424509491 for osd.6 2024-09-18T01:01:17.446 DEBUG:teuthology.parallel:result is None 2024-09-18T01:01:17.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509493 for osd.2 2024-09-18T01:01:17.451 DEBUG:teuthology.parallel:result is None 2024-09-18T01:01:17.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509490 got 64424509490 for osd.5 2024-09-18T01:01:17.457 DEBUG:teuthology.parallel:result is None 2024-09-18T01:01:17.463 INFO:teuthology.orchestra.run.smithi042.stdout:64424509492 2024-09-18T01:01:17.464 INFO:teuthology.orchestra.run.smithi042.stdout:64424509492 2024-09-18T01:01:17.479 INFO:teuthology.orchestra.run.smithi042.stdout:55834574899 2024-09-18T01:01:17.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509492 for osd.1 2024-09-18T01:01:17.491 DEBUG:teuthology.parallel:result is None 2024-09-18T01:01:17.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509492 for osd.0 2024-09-18T01:01:17.492 DEBUG:teuthology.parallel:result is None 2024-09-18T01:01:17.492 INFO:teuthology.orchestra.run.smithi042.stdout:64424509492 2024-09-18T01:01:17.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574899 for osd.7 2024-09-18T01:01:17.499 DEBUG:teuthology.parallel:result is None 2024-09-18T01:01:17.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509492 for osd.3 2024-09-18T01:01:17.510 DEBUG:teuthology.parallel:result is None 2024-09-18T01:01:17.510 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:01:17.510 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:01:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:01:17.841 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T01:01:17.857 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-18T01:01:15.945362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":482,"num_read":220710,"num_read_kb":1067391,"num_write":22422,"num_write_kb":690995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12021,"num_bytes_recovered":693846218,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19658,"ondisk_log_size":19658,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423528,"kb_used_data":21300,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754551192,"statfs":{"total":773094113280,"available":772660420608,"internally_reserved":0,"allocated":21811200,"data_stored":11109709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.600595"},"pg_stats":[{"pgid":"3.1f","version":"186'621","reported_seq":4221,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.577081+0000","last_change":"2024-09-18T01:00:44.155176+0000","last_active":"2024-09-18T01:01:13.577081+0000","last_peered":"2024-09-18T01:01:13.577081+0000","last_clean":"2024-09-18T01:01:13.577081+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T01:01:13.577081+0000","last_undegraded":"2024-09-18T01:01:13.577081+0000","last_fullsized":"2024-09-18T01:01:13.577081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:49.598474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001427074,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6437,"num_read_kb":23165,"num_write":614,"num_write_kb":11006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"186'646","reported_seq":5757,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.577094+0000","last_change":"2024-09-18T01:01:01.248306+0000","last_active":"2024-09-18T01:01:13.577094+0000","last_peered":"2024-09-18T01:01:13.577094+0000","last_clean":"2024-09-18T01:01:13.577094+0000","last_became_active":"2024-09-18T01:01:01.245192+0000","last_became_peered":"2024-09-18T01:01:01.245192+0000","last_unstale":"2024-09-18T01:01:13.577094+0000","last_undegraded":"2024-09-18T01:01:13.577094+0000","last_fullsized":"2024-09-18T01:01:13.577094+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:50.236589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041620060000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4822,"num_read_kb":36392,"num_write":703,"num_write_kb":11559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":30825045,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"184'800","reported_seq":12971,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382040+0000","last_change":"2024-09-18T01:01:03.959251+0000","last_active":"2024-09-18T01:01:14.382040+0000","last_peered":"2024-09-18T01:01:14.382040+0000","last_clean":"2024-09-18T01:01:14.382040+0000","last_became_active":"2024-09-18T01:01:03.956061+0000","last_became_peered":"2024-09-18T01:01:03.956061+0000","last_unstale":"2024-09-18T01:01:14.382040+0000","last_undegraded":"2024-09-18T01:01:14.382040+0000","last_fullsized":"2024-09-18T01:01:14.382040+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:21:07.686876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034116569999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":13334,"num_read_kb":40113,"num_write":945,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":705,"num_bytes_recovered":22094834,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"186'663","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912412+0000","last_change":"2024-09-18T01:01:01.947287+0000","last_active":"2024-09-18T01:01:14.912412+0000","last_peered":"2024-09-18T01:01:14.912412+0000","last_clean":"2024-09-18T01:01:14.912412+0000","last_became_active":"2024-09-18T01:01:01.945747+0000","last_became_peered":"2024-09-18T01:01:01.945747+0000","last_unstale":"2024-09-18T01:01:14.912412+0000","last_undegraded":"2024-09-18T01:01:14.912412+0000","last_fullsized":"2024-09-18T01:01:14.912412+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:44:09.776110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002343262,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5693,"num_read_kb":28138,"num_write":736,"num_write_kb":11405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":5428683,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"186'1093","reported_seq":10874,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.365175+0000","last_change":"2024-09-18T01:01:01.953195+0000","last_active":"2024-09-18T01:01:13.365175+0000","last_peered":"2024-09-18T01:01:13.365175+0000","last_clean":"2024-09-18T01:01:13.365175+0000","last_became_active":"2024-09-18T01:01:01.947957+0000","last_became_peered":"2024-09-18T01:01:01.947957+0000","last_unstale":"2024-09-18T01:01:13.365175+0000","last_undegraded":"2024-09-18T01:01:13.365175+0000","last_fullsized":"2024-09-18T01:01:13.365175+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:50:57.100165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003279527,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12686,"num_read_kb":40787,"num_write":1375,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":59340116,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"186'364","reported_seq":5144,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912518+0000","last_change":"2024-09-18T01:00:44.151068+0000","last_active":"2024-09-18T01:01:14.912518+0000","last_peered":"2024-09-18T01:01:14.912518+0000","last_clean":"2024-09-18T01:01:14.912518+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T01:01:14.912518+0000","last_undegraded":"2024-09-18T01:01:14.912518+0000","last_fullsized":"2024-09-18T01:01:14.912518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:24.750570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017298630000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5410,"num_read_kb":29581,"num_write":319,"num_write_kb":12768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"186'701","reported_seq":30886,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.365368+0000","last_change":"2024-09-18T01:00:51.958851+0000","last_active":"2024-09-18T01:01:13.365368+0000","last_peered":"2024-09-18T01:01:13.365368+0000","last_clean":"2024-09-18T01:01:13.365368+0000","last_became_active":"2024-09-18T01:00:51.953781+0000","last_became_peered":"2024-09-18T01:00:51.953781+0000","last_unstale":"2024-09-18T01:01:13.365368+0000","last_undegraded":"2024-09-18T01:01:13.365368+0000","last_fullsized":"2024-09-18T01:01:13.365368+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:44.107525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001459178,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":30614,"num_read_kb":78260,"num_write":886,"num_write_kb":27079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":40291284,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"186'656","reported_seq":4015,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.577050+0000","last_change":"2024-09-18T01:00:44.157985+0000","last_active":"2024-09-18T01:01:13.577050+0000","last_peered":"2024-09-18T01:01:13.577050+0000","last_clean":"2024-09-18T01:01:13.577050+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T01:01:13.577050+0000","last_undegraded":"2024-09-18T01:01:13.577050+0000","last_fullsized":"2024-09-18T01:01:13.577050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:27.075227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039694869999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4869,"num_read_kb":13814,"num_write":877,"num_write_kb":42194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"185'676","reported_seq":7353,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.577203+0000","last_change":"2024-09-18T01:00:44.156705+0000","last_active":"2024-09-18T01:01:13.577203+0000","last_peered":"2024-09-18T01:01:13.577203+0000","last_clean":"2024-09-18T01:01:13.577203+0000","last_became_active":"2024-09-18T01:00:20.017711+0000","last_became_peered":"2024-09-18T01:00:20.017711+0000","last_unstale":"2024-09-18T01:01:13.577203+0000","last_undegraded":"2024-09-18T01:01:13.577203+0000","last_fullsized":"2024-09-18T01:01:13.577203+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:49.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028584439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7284,"num_read_kb":46982,"num_write":773,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":13346337,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"186'641","reported_seq":8980,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380983+0000","last_change":"2024-09-18T01:01:01.955368+0000","last_active":"2024-09-18T01:01:14.380983+0000","last_peered":"2024-09-18T01:01:14.380983+0000","last_clean":"2024-09-18T01:01:14.380983+0000","last_became_active":"2024-09-18T01:01:01.952512+0000","last_became_peered":"2024-09-18T01:01:01.952512+0000","last_unstale":"2024-09-18T01:01:14.380983+0000","last_undegraded":"2024-09-18T01:01:14.380983+0000","last_fullsized":"2024-09-18T01:01:14.380983+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:36:14.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887849999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8507,"num_read_kb":37252,"num_write":699,"num_write_kb":22150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":946,"num_bytes_recovered":37601010,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"184'519","reported_seq":3007,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941255+0000","last_change":"2024-09-18T01:01:02.959624+0000","last_active":"2024-09-18T01:01:14.941255+0000","last_peered":"2024-09-18T01:01:14.941255+0000","last_clean":"2024-09-18T01:01:14.941255+0000","last_became_active":"2024-09-18T01:01:02.956655+0000","last_became_peered":"2024-09-18T01:01:02.956655+0000","last_unstale":"2024-09-18T01:01:14.941255+0000","last_undegraded":"2024-09-18T01:01:14.941255+0000","last_fullsized":"2024-09-18T01:01:14.941255+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:53:56.284343+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010838409,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2322,"num_read_kb":15300,"num_write":809,"num_write_kb":48070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":22291504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"181'371","reported_seq":3210,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381490+0000","last_change":"2024-09-18T01:01:01.948935+0000","last_active":"2024-09-18T01:01:14.381490+0000","last_peered":"2024-09-18T01:01:14.381490+0000","last_clean":"2024-09-18T01:01:14.381490+0000","last_became_active":"2024-09-18T01:01:01.947158+0000","last_became_peered":"2024-09-18T01:01:01.947158+0000","last_unstale":"2024-09-18T01:01:14.381490+0000","last_undegraded":"2024-09-18T01:01:14.381490+0000","last_fullsized":"2024-09-18T01:01:14.381490+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:05.169088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013361650000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2452,"num_read_kb":3816,"num_write":339,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":3474495,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":127,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382692+0000","last_change":"2024-09-18T01:00:59.238970+0000","last_active":"2024-09-18T01:01:14.382692+0000","last_peered":"2024-09-18T01:01:14.382692+0000","last_clean":"2024-09-18T01:01:14.382692+0000","last_became_active":"2024-09-18T01:00:59.238106+0000","last_became_peered":"2024-09-18T01:00:59.238106+0000","last_unstale":"2024-09-18T01:01:14.382692+0000","last_undegraded":"2024-09-18T01:01:14.382692+0000","last_fullsized":"2024-09-18T01:01:14.382692+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:15:05.506655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"186'777","reported_seq":4881,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.365470+0000","last_change":"2024-09-18T01:00:44.148713+0000","last_active":"2024-09-18T01:01:13.365470+0000","last_peered":"2024-09-18T01:01:13.365470+0000","last_clean":"2024-09-18T01:01:13.365470+0000","last_became_active":"2024-09-18T01:00:11.936218+0000","last_became_peered":"2024-09-18T01:00:11.936218+0000","last_unstale":"2024-09-18T01:01:13.365470+0000","last_undegraded":"2024-09-18T01:01:13.365470+0000","last_fullsized":"2024-09-18T01:01:13.365470+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:24.939297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002351681,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5488,"num_read_kb":22949,"num_write":1002,"num_write_kb":4650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":2805113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":410,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.365456+0000","last_change":"2024-09-18T01:00:59.233973+0000","last_active":"2024-09-18T01:01:13.365456+0000","last_peered":"2024-09-18T01:01:13.365456+0000","last_clean":"2024-09-18T01:01:13.365456+0000","last_became_active":"2024-09-18T01:00:59.231756+0000","last_became_peered":"2024-09-18T01:00:59.231756+0000","last_unstale":"2024-09-18T01:01:13.365456+0000","last_undegraded":"2024-09-18T01:01:13.365456+0000","last_fullsized":"2024-09-18T01:01:13.365456+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:50:54.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"185'506","reported_seq":8881,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.365268+0000","last_change":"2024-09-18T01:00:44.150561+0000","last_active":"2024-09-18T01:01:13.365268+0000","last_peered":"2024-09-18T01:01:13.365268+0000","last_clean":"2024-09-18T01:01:13.365268+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T01:01:13.365268+0000","last_undegraded":"2024-09-18T01:01:13.365268+0000","last_fullsized":"2024-09-18T01:01:13.365268+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:44.745709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036716330000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8918,"num_read_kb":44936,"num_write":410,"num_write_kb":13864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382260+0000","last_change":"2024-09-18T01:00:15.991240+0000","last_active":"2024-09-18T01:01:14.382260+0000","last_peered":"2024-09-18T01:01:14.382260+0000","last_clean":"2024-09-18T01:01:14.382260+0000","last_became_active":"2024-09-18T01:00:15.990930+0000","last_became_peered":"2024-09-18T01:00:15.990930+0000","last_unstale":"2024-09-18T01:01:14.382260+0000","last_undegraded":"2024-09-18T01:01:14.382260+0000","last_fullsized":"2024-09-18T01:01:14.382260+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:55.616429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"185'348","reported_seq":4491,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173966+0000","last_change":"2024-09-18T01:00:44.260854+0000","last_active":"2024-09-18T01:01:15.173966+0000","last_peered":"2024-09-18T01:01:15.173966+0000","last_clean":"2024-09-18T01:01:15.173966+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T01:01:15.173966+0000","last_undegraded":"2024-09-18T01:01:15.173966+0000","last_fullsized":"2024-09-18T01:01:15.173966+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:09.012851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019390989999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3644,"num_read_kb":27224,"num_write":289,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":397,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.576963+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T01:01:13.576963+0000","last_peered":"2024-09-18T01:01:13.576963+0000","last_clean":"2024-09-18T01:01:13.576963+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T01:01:13.576963+0000","last_undegraded":"2024-09-18T01:01:13.576963+0000","last_fullsized":"2024-09-18T01:01:13.576963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"185'580","reported_seq":4494,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380809+0000","last_change":"2024-09-18T01:01:01.242483+0000","last_active":"2024-09-18T01:01:14.380809+0000","last_peered":"2024-09-18T01:01:14.380809+0000","last_clean":"2024-09-18T01:01:14.380809+0000","last_became_active":"2024-09-18T01:01:01.239957+0000","last_became_peered":"2024-09-18T01:01:01.239957+0000","last_unstale":"2024-09-18T01:01:14.380809+0000","last_undegraded":"2024-09-18T01:01:14.380809+0000","last_fullsized":"2024-09-18T01:01:14.380809+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:46.781823+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.024809728,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4830,"num_read_kb":33562,"num_write":800,"num_write_kb":24925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"186'946","reported_seq":6143,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941033+0000","last_change":"2024-09-18T01:01:03.958936+0000","last_active":"2024-09-18T01:01:14.941033+0000","last_peered":"2024-09-18T01:01:14.941033+0000","last_clean":"2024-09-18T01:01:14.941033+0000","last_became_active":"2024-09-18T01:01:03.955764+0000","last_became_peered":"2024-09-18T01:01:03.955764+0000","last_unstale":"2024-09-18T01:01:14.941033+0000","last_undegraded":"2024-09-18T01:01:14.941033+0000","last_fullsized":"2024-09-18T01:01:14.941033+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:25.024002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022401439999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7644,"num_read_kb":19178,"num_write":1076,"num_write_kb":33944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":99850853,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":56,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380143+0000","last_change":"2024-09-18T01:00:51.188692+0000","last_active":"2024-09-18T01:01:14.380143+0000","last_peered":"2024-09-18T01:01:14.380143+0000","last_clean":"2024-09-18T01:01:14.380143+0000","last_became_active":"2024-09-18T01:00:51.183713+0000","last_became_peered":"2024-09-18T01:00:51.183713+0000","last_unstale":"2024-09-18T01:01:14.380143+0000","last_undegraded":"2024-09-18T01:01:14.380143+0000","last_fullsized":"2024-09-18T01:01:14.380143+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:20:22.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"185'657","reported_seq":8380,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.576740+0000","last_change":"2024-09-18T01:00:44.156905+0000","last_active":"2024-09-18T01:01:13.576740+0000","last_peered":"2024-09-18T01:01:13.576740+0000","last_clean":"2024-09-18T01:01:13.576740+0000","last_became_active":"2024-09-18T01:00:08.951992+0000","last_became_peered":"2024-09-18T01:00:08.951992+0000","last_unstale":"2024-09-18T01:01:13.576740+0000","last_undegraded":"2024-09-18T01:01:13.576740+0000","last_fullsized":"2024-09-18T01:01:13.576740+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:59.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030555019999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7663,"num_read_kb":29924,"num_write":748,"num_write_kb":19563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":15190015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381997+0000","last_change":"2024-09-18T01:00:04.976725+0000","last_active":"2024-09-18T01:01:14.381997+0000","last_peered":"2024-09-18T01:01:14.381997+0000","last_clean":"2024-09-18T01:01:14.381997+0000","last_became_active":"2024-09-18T01:00:04.976360+0000","last_became_peered":"2024-09-18T01:00:04.976360+0000","last_unstale":"2024-09-18T01:01:14.381997+0000","last_undegraded":"2024-09-18T01:01:14.381997+0000","last_fullsized":"2024-09-18T01:01:14.381997+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:11.159126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"184'548","reported_seq":9961,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173598+0000","last_change":"2024-09-18T01:01:01.957077+0000","last_active":"2024-09-18T01:01:15.173598+0000","last_peered":"2024-09-18T01:01:15.173598+0000","last_clean":"2024-09-18T01:01:15.173598+0000","last_became_active":"2024-09-18T01:01:01.953404+0000","last_became_peered":"2024-09-18T01:01:01.953404+0000","last_unstale":"2024-09-18T01:01:15.173598+0000","last_undegraded":"2024-09-18T01:01:15.173598+0000","last_fullsized":"2024-09-18T01:01:15.173598+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:03.050834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726180000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8919,"num_read_kb":48544,"num_write":559,"num_write_kb":18757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1056,"num_bytes_recovered":19034719,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379823+0000","last_change":"2024-09-18T01:00:15.994517+0000","last_active":"2024-09-18T01:01:14.379823+0000","last_peered":"2024-09-18T01:01:14.379823+0000","last_clean":"2024-09-18T01:01:14.379823+0000","last_became_active":"2024-09-18T01:00:15.994159+0000","last_became_peered":"2024-09-18T01:00:15.994159+0000","last_unstale":"2024-09-18T01:01:14.379823+0000","last_undegraded":"2024-09-18T01:01:14.379823+0000","last_fullsized":"2024-09-18T01:01:14.379823+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:06.775713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"186'722","reported_seq":5330,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.365198+0000","last_change":"2024-09-18T01:01:01.952665+0000","last_active":"2024-09-18T01:01:13.365198+0000","last_peered":"2024-09-18T01:01:13.365198+0000","last_clean":"2024-09-18T01:01:13.365198+0000","last_became_active":"2024-09-18T01:01:01.948535+0000","last_became_peered":"2024-09-18T01:01:01.948535+0000","last_unstale":"2024-09-18T01:01:13.365198+0000","last_undegraded":"2024-09-18T01:01:13.365198+0000","last_fullsized":"2024-09-18T01:01:13.365198+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:39.959837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00139405,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5240,"num_read_kb":16439,"num_write":1146,"num_write_kb":52135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":12961092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":153,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380940+0000","last_change":"2024-09-18T01:00:58.232332+0000","last_active":"2024-09-18T01:01:14.380940+0000","last_peered":"2024-09-18T01:01:14.380940+0000","last_clean":"2024-09-18T01:01:14.380940+0000","last_became_active":"2024-09-18T01:00:58.229581+0000","last_became_peered":"2024-09-18T01:00:58.229581+0000","last_unstale":"2024-09-18T01:01:14.380940+0000","last_undegraded":"2024-09-18T01:01:14.380940+0000","last_fullsized":"2024-09-18T01:01:14.380940+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:05:38.818642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":551,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912328+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T01:01:14.912328+0000","last_peered":"2024-09-18T01:01:14.912328+0000","last_clean":"2024-09-18T01:01:14.912328+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T01:01:14.912328+0000","last_undegraded":"2024-09-18T01:01:14.912328+0000","last_fullsized":"2024-09-18T01:01:14.912328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"184'448","reported_seq":5877,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-18T01:01:13.365071+0000","last_change":"2024-09-18T01:00:44.149901+0000","last_active":"2024-09-18T01:01:13.365071+0000","last_peered":"2024-09-18T01:01:13.365071+0000","last_clean":"2024-09-18T01:01:13.365071+0000","last_became_active":"2024-09-18T01:00:08.949384+0000","last_became_peered":"2024-09-18T01:00:08.949384+0000","last_unstale":"2024-09-18T01:01:13.365071+0000","last_undegraded":"2024-09-18T01:01:13.365071+0000","last_fullsized":"2024-09-18T01:01:13.365071+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:20.064612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033952990000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5722,"num_read_kb":51790,"num_write":458,"num_write_kb":39546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":32435091,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"185'408","reported_seq":3782,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379828+0000","last_change":"2024-09-18T01:00:44.145255+0000","last_active":"2024-09-18T01:01:14.379828+0000","last_peered":"2024-09-18T01:01:14.379828+0000","last_clean":"2024-09-18T01:01:14.379828+0000","last_became_active":"2024-09-18T01:00:18.997710+0000","last_became_peered":"2024-09-18T01:00:18.997710+0000","last_unstale":"2024-09-18T01:01:14.379828+0000","last_undegraded":"2024-09-18T01:01:14.379828+0000","last_fullsized":"2024-09-18T01:01:14.379828+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:36.454597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016382809999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3625,"num_read_kb":11553,"num_write":342,"num_write_kb":23969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":27148173,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"185'607","reported_seq":5628,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380626+0000","last_change":"2024-09-18T01:00:44.146011+0000","last_active":"2024-09-18T01:01:14.380626+0000","last_peered":"2024-09-18T01:01:14.380626+0000","last_clean":"2024-09-18T01:01:14.380626+0000","last_became_active":"2024-09-18T01:00:22.938344+0000","last_became_peered":"2024-09-18T01:00:22.938344+0000","last_unstale":"2024-09-18T01:01:14.380626+0000","last_undegraded":"2024-09-18T01:01:14.380626+0000","last_fullsized":"2024-09-18T01:01:14.380626+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:59:54.570595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021924589999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4754,"num_read_kb":46819,"num_write":596,"num_write_kb":17196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":6639875,"num_keys_recovered":85,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"185'515","reported_seq":5914,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382003+0000","last_change":"2024-09-18T01:00:44.262988+0000","last_active":"2024-09-18T01:01:14.382003+0000","last_peered":"2024-09-18T01:01:14.382003+0000","last_clean":"2024-09-18T01:01:14.382003+0000","last_became_active":"2024-09-18T01:00:10.954910+0000","last_became_peered":"2024-09-18T01:00:10.954910+0000","last_unstale":"2024-09-18T01:01:14.382003+0000","last_undegraded":"2024-09-18T01:01:14.382003+0000","last_fullsized":"2024-09-18T01:01:14.382003+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:45.069175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042875949999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5466,"num_read_kb":66445,"num_write":496,"num_write_kb":15031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15399944,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"185'566","reported_seq":7107,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941155+0000","last_change":"2024-09-18T01:01:02.956425+0000","last_active":"2024-09-18T01:01:14.941155+0000","last_peered":"2024-09-18T01:01:14.941155+0000","last_clean":"2024-09-18T01:01:14.941155+0000","last_became_active":"2024-09-18T01:01:02.953950+0000","last_became_peered":"2024-09-18T01:01:02.953950+0000","last_unstale":"2024-09-18T01:01:14.941155+0000","last_undegraded":"2024-09-18T01:01:14.941155+0000","last_fullsized":"2024-09-18T01:01:14.941155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:11.064860+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0026394880000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6401,"num_read_kb":47394,"num_write":633,"num_write_kb":22377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":560,"num_bytes_recovered":26783458,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"184'976","reported_seq":6804,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381748+0000","last_change":"2024-09-18T01:00:44.146438+0000","last_active":"2024-09-18T01:01:14.381748+0000","last_peered":"2024-09-18T01:01:14.381748+0000","last_clean":"2024-09-18T01:01:14.381748+0000","last_became_active":"2024-09-18T01:00:21.923279+0000","last_became_peered":"2024-09-18T01:00:21.923279+0000","last_unstale":"2024-09-18T01:01:14.381748+0000","last_undegraded":"2024-09-18T01:01:14.381748+0000","last_fullsized":"2024-09-18T01:01:14.381748+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:23.093467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026100659999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8636,"num_read_kb":40955,"num_write":1143,"num_write_kb":30743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":80305919,"num_keys_recovered":163,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"183'713","reported_seq":4745,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380694+0000","last_change":"2024-09-18T01:01:02.959668+0000","last_active":"2024-09-18T01:01:14.380694+0000","last_peered":"2024-09-18T01:01:14.380694+0000","last_clean":"2024-09-18T01:01:14.380694+0000","last_became_active":"2024-09-18T01:01:02.954337+0000","last_became_peered":"2024-09-18T01:01:02.954337+0000","last_unstale":"2024-09-18T01:01:14.380694+0000","last_undegraded":"2024-09-18T01:01:14.380694+0000","last_fullsized":"2024-09-18T01:01:14.380694+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:56:25.045514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0093305890000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4741,"num_read_kb":29100,"num_write":948,"num_write_kb":26466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":41587165,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"184'563","reported_seq":4380,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941359+0000","last_change":"2024-09-18T01:01:01.950143+0000","last_active":"2024-09-18T01:01:14.941359+0000","last_peered":"2024-09-18T01:01:14.941359+0000","last_clean":"2024-09-18T01:01:14.941359+0000","last_became_active":"2024-09-18T01:01:01.948267+0000","last_became_peered":"2024-09-18T01:01:01.948267+0000","last_unstale":"2024-09-18T01:01:14.941359+0000","last_undegraded":"2024-09-18T01:01:14.941359+0000","last_fullsized":"2024-09-18T01:01:14.941359+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:43:18.359056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024190230000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5311,"num_read_kb":28191,"num_write":598,"num_write_kb":9287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6320640,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"184'456","reported_seq":5302,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912243+0000","last_change":"2024-09-18T01:00:44.148986+0000","last_active":"2024-09-18T01:01:14.912243+0000","last_peered":"2024-09-18T01:01:14.912243+0000","last_clean":"2024-09-18T01:01:14.912243+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T01:01:14.912243+0000","last_undegraded":"2024-09-18T01:01:14.912243+0000","last_fullsized":"2024-09-18T01:01:14.912243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:48:20.609462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001443239,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":51584,"num_write":357,"num_write_kb":26802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"183'292","reported_seq":3444,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380096+0000","last_change":"2024-09-18T01:01:02.956466+0000","last_active":"2024-09-18T01:01:14.380096+0000","last_peered":"2024-09-18T01:01:14.380096+0000","last_clean":"2024-09-18T01:01:14.380096+0000","last_became_active":"2024-09-18T01:01:02.950361+0000","last_became_peered":"2024-09-18T01:01:02.950361+0000","last_unstale":"2024-09-18T01:01:14.380096+0000","last_undegraded":"2024-09-18T01:01:14.380096+0000","last_fullsized":"2024-09-18T01:01:14.380096+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:56:04.680353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033253200000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1897,"num_read_kb":13175,"num_write":246,"num_write_kb":14260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":15824068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"184'638","reported_seq":6909,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173643+0000","last_change":"2024-09-18T01:00:44.261997+0000","last_active":"2024-09-18T01:01:15.173643+0000","last_peered":"2024-09-18T01:01:15.173643+0000","last_clean":"2024-09-18T01:01:15.173643+0000","last_became_active":"2024-09-18T01:00:21.926814+0000","last_became_peered":"2024-09-18T01:00:21.926814+0000","last_unstale":"2024-09-18T01:01:15.173643+0000","last_undegraded":"2024-09-18T01:01:15.173643+0000","last_fullsized":"2024-09-18T01:01:15.173643+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:04.145432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002915932,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6954,"num_read_kb":17130,"num_write":754,"num_write_kb":11285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":32804395,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"185'521","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381219+0000","last_change":"2024-09-18T01:01:01.245238+0000","last_active":"2024-09-18T01:01:14.381219+0000","last_peered":"2024-09-18T01:01:14.381219+0000","last_clean":"2024-09-18T01:01:14.381219+0000","last_became_active":"2024-09-18T01:01:01.242541+0000","last_became_peered":"2024-09-18T01:01:01.242541+0000","last_unstale":"2024-09-18T01:01:14.381219+0000","last_undegraded":"2024-09-18T01:01:14.381219+0000","last_fullsized":"2024-09-18T01:01:14.381219+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:44.647014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027658140000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5114,"num_read_kb":26799,"num_write":585,"num_write_kb":11735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":9379374,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":482,"num_read":220594,"num_read_kb":1067291,"num_write":22261,"num_write_kb":688577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12013,"num_bytes_recovered":693846180,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19538,"ondisk_log_size":19538,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47592,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324248,"statfs":{"total":96636764160,"available":96588029952,"internally_reserved":0,"allocated":2572288,"data_stored":1237498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509490,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48168,"kb_used_data":3092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323672,"statfs":{"total":96636764160,"available":96587440128,"internally_reserved":0,"allocated":3166208,"data_stored":1827866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60064,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311776,"statfs":{"total":96636764160,"available":96575258624,"internally_reserved":0,"allocated":2568192,"data_stored":1237479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509492,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55920,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315920,"statfs":{"total":96636764160,"available":96579502080,"internally_reserved":0,"allocated":2584576,"data_stored":1243311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60080,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311760,"statfs":{"total":96636764160,"available":96575242240,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52344,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319496,"statfs":{"total":96636764160,"available":96583163904,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47600,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324240,"statfs":{"total":96636764160,"available":96588021760,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:01:17.859 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:01:18.145 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:01:18.146 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T01:01:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-18T01:01:17.946193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":482,"num_read":220710,"num_read_kb":1067391,"num_write":22422,"num_write_kb":690995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12021,"num_bytes_recovered":693846218,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19658,"ondisk_log_size":19658,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":427704,"kb_used_data":21312,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754547016,"statfs":{"total":773094113280,"available":772656144384,"internally_reserved":0,"allocated":21823488,"data_stored":11115522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":415944240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001933"},"pg_stats":[{"pgid":"3.1f","version":"186'621","reported_seq":4223,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590664+0000","last_change":"2024-09-18T01:00:44.155176+0000","last_active":"2024-09-18T01:01:15.590664+0000","last_peered":"2024-09-18T01:01:15.590664+0000","last_clean":"2024-09-18T01:01:15.590664+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T01:01:15.590664+0000","last_undegraded":"2024-09-18T01:01:15.590664+0000","last_fullsized":"2024-09-18T01:01:15.590664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:49.598474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001427074,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6437,"num_read_kb":23165,"num_write":614,"num_write_kb":11006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"186'646","reported_seq":5759,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590628+0000","last_change":"2024-09-18T01:01:01.248306+0000","last_active":"2024-09-18T01:01:15.590628+0000","last_peered":"2024-09-18T01:01:15.590628+0000","last_clean":"2024-09-18T01:01:15.590628+0000","last_became_active":"2024-09-18T01:01:01.245192+0000","last_became_peered":"2024-09-18T01:01:01.245192+0000","last_unstale":"2024-09-18T01:01:15.590628+0000","last_undegraded":"2024-09-18T01:01:15.590628+0000","last_fullsized":"2024-09-18T01:01:15.590628+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:50.236589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041620060000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4822,"num_read_kb":36392,"num_write":703,"num_write_kb":11559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":30825045,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"184'800","reported_seq":12971,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382040+0000","last_change":"2024-09-18T01:01:03.959251+0000","last_active":"2024-09-18T01:01:14.382040+0000","last_peered":"2024-09-18T01:01:14.382040+0000","last_clean":"2024-09-18T01:01:14.382040+0000","last_became_active":"2024-09-18T01:01:03.956061+0000","last_became_peered":"2024-09-18T01:01:03.956061+0000","last_unstale":"2024-09-18T01:01:14.382040+0000","last_undegraded":"2024-09-18T01:01:14.382040+0000","last_fullsized":"2024-09-18T01:01:14.382040+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:21:07.686876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034116569999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":13334,"num_read_kb":40113,"num_write":945,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":705,"num_bytes_recovered":22094834,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"186'663","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912412+0000","last_change":"2024-09-18T01:01:01.947287+0000","last_active":"2024-09-18T01:01:14.912412+0000","last_peered":"2024-09-18T01:01:14.912412+0000","last_clean":"2024-09-18T01:01:14.912412+0000","last_became_active":"2024-09-18T01:01:01.945747+0000","last_became_peered":"2024-09-18T01:01:01.945747+0000","last_unstale":"2024-09-18T01:01:14.912412+0000","last_undegraded":"2024-09-18T01:01:14.912412+0000","last_fullsized":"2024-09-18T01:01:14.912412+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:44:09.776110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002343262,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5693,"num_read_kb":28138,"num_write":736,"num_write_kb":11405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":5428683,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"186'1093","reported_seq":10876,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340934+0000","last_change":"2024-09-18T01:01:01.953195+0000","last_active":"2024-09-18T01:01:15.340934+0000","last_peered":"2024-09-18T01:01:15.340934+0000","last_clean":"2024-09-18T01:01:15.340934+0000","last_became_active":"2024-09-18T01:01:01.947957+0000","last_became_peered":"2024-09-18T01:01:01.947957+0000","last_unstale":"2024-09-18T01:01:15.340934+0000","last_undegraded":"2024-09-18T01:01:15.340934+0000","last_fullsized":"2024-09-18T01:01:15.340934+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:50:57.100165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003279527,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12686,"num_read_kb":40787,"num_write":1375,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":59340116,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"186'364","reported_seq":5144,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912518+0000","last_change":"2024-09-18T01:00:44.151068+0000","last_active":"2024-09-18T01:01:14.912518+0000","last_peered":"2024-09-18T01:01:14.912518+0000","last_clean":"2024-09-18T01:01:14.912518+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T01:01:14.912518+0000","last_undegraded":"2024-09-18T01:01:14.912518+0000","last_fullsized":"2024-09-18T01:01:14.912518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:24.750570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017298630000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5410,"num_read_kb":29581,"num_write":319,"num_write_kb":12768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"186'701","reported_seq":30888,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340790+0000","last_change":"2024-09-18T01:00:51.958851+0000","last_active":"2024-09-18T01:01:15.340790+0000","last_peered":"2024-09-18T01:01:15.340790+0000","last_clean":"2024-09-18T01:01:15.340790+0000","last_became_active":"2024-09-18T01:00:51.953781+0000","last_became_peered":"2024-09-18T01:00:51.953781+0000","last_unstale":"2024-09-18T01:01:15.340790+0000","last_undegraded":"2024-09-18T01:01:15.340790+0000","last_fullsized":"2024-09-18T01:01:15.340790+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:44.107525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001459178,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":30614,"num_read_kb":78260,"num_write":886,"num_write_kb":27079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":40291284,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"186'656","reported_seq":4017,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.591127+0000","last_change":"2024-09-18T01:00:44.157985+0000","last_active":"2024-09-18T01:01:15.591127+0000","last_peered":"2024-09-18T01:01:15.591127+0000","last_clean":"2024-09-18T01:01:15.591127+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T01:01:15.591127+0000","last_undegraded":"2024-09-18T01:01:15.591127+0000","last_fullsized":"2024-09-18T01:01:15.591127+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:27.075227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039694869999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4869,"num_read_kb":13814,"num_write":877,"num_write_kb":42194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"185'676","reported_seq":7355,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590783+0000","last_change":"2024-09-18T01:00:44.156705+0000","last_active":"2024-09-18T01:01:15.590783+0000","last_peered":"2024-09-18T01:01:15.590783+0000","last_clean":"2024-09-18T01:01:15.590783+0000","last_became_active":"2024-09-18T01:00:20.017711+0000","last_became_peered":"2024-09-18T01:00:20.017711+0000","last_unstale":"2024-09-18T01:01:15.590783+0000","last_undegraded":"2024-09-18T01:01:15.590783+0000","last_fullsized":"2024-09-18T01:01:15.590783+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:49.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028584439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7284,"num_read_kb":46982,"num_write":773,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":13346337,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"186'641","reported_seq":8980,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380983+0000","last_change":"2024-09-18T01:01:01.955368+0000","last_active":"2024-09-18T01:01:14.380983+0000","last_peered":"2024-09-18T01:01:14.380983+0000","last_clean":"2024-09-18T01:01:14.380983+0000","last_became_active":"2024-09-18T01:01:01.952512+0000","last_became_peered":"2024-09-18T01:01:01.952512+0000","last_unstale":"2024-09-18T01:01:14.380983+0000","last_undegraded":"2024-09-18T01:01:14.380983+0000","last_fullsized":"2024-09-18T01:01:14.380983+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:36:14.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887849999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8507,"num_read_kb":37252,"num_write":699,"num_write_kb":22150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":946,"num_bytes_recovered":37601010,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"184'519","reported_seq":3007,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941255+0000","last_change":"2024-09-18T01:01:02.959624+0000","last_active":"2024-09-18T01:01:14.941255+0000","last_peered":"2024-09-18T01:01:14.941255+0000","last_clean":"2024-09-18T01:01:14.941255+0000","last_became_active":"2024-09-18T01:01:02.956655+0000","last_became_peered":"2024-09-18T01:01:02.956655+0000","last_unstale":"2024-09-18T01:01:14.941255+0000","last_undegraded":"2024-09-18T01:01:14.941255+0000","last_fullsized":"2024-09-18T01:01:14.941255+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:53:56.284343+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010838409,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2322,"num_read_kb":15300,"num_write":809,"num_write_kb":48070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":22291504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"181'371","reported_seq":3210,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381490+0000","last_change":"2024-09-18T01:01:01.948935+0000","last_active":"2024-09-18T01:01:14.381490+0000","last_peered":"2024-09-18T01:01:14.381490+0000","last_clean":"2024-09-18T01:01:14.381490+0000","last_became_active":"2024-09-18T01:01:01.947158+0000","last_became_peered":"2024-09-18T01:01:01.947158+0000","last_unstale":"2024-09-18T01:01:14.381490+0000","last_undegraded":"2024-09-18T01:01:14.381490+0000","last_fullsized":"2024-09-18T01:01:14.381490+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:05.169088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013361650000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2452,"num_read_kb":3816,"num_write":339,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":3474495,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":127,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382692+0000","last_change":"2024-09-18T01:00:59.238970+0000","last_active":"2024-09-18T01:01:14.382692+0000","last_peered":"2024-09-18T01:01:14.382692+0000","last_clean":"2024-09-18T01:01:14.382692+0000","last_became_active":"2024-09-18T01:00:59.238106+0000","last_became_peered":"2024-09-18T01:00:59.238106+0000","last_unstale":"2024-09-18T01:01:14.382692+0000","last_undegraded":"2024-09-18T01:01:14.382692+0000","last_fullsized":"2024-09-18T01:01:14.382692+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:15:05.506655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"186'777","reported_seq":4883,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.341002+0000","last_change":"2024-09-18T01:00:44.148713+0000","last_active":"2024-09-18T01:01:15.341002+0000","last_peered":"2024-09-18T01:01:15.341002+0000","last_clean":"2024-09-18T01:01:15.341002+0000","last_became_active":"2024-09-18T01:00:11.936218+0000","last_became_peered":"2024-09-18T01:00:11.936218+0000","last_unstale":"2024-09-18T01:01:15.341002+0000","last_undegraded":"2024-09-18T01:01:15.341002+0000","last_fullsized":"2024-09-18T01:01:15.341002+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:24.939297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002351681,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5488,"num_read_kb":22949,"num_write":1002,"num_write_kb":4650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":2805113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":412,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340720+0000","last_change":"2024-09-18T01:00:59.233973+0000","last_active":"2024-09-18T01:01:15.340720+0000","last_peered":"2024-09-18T01:01:15.340720+0000","last_clean":"2024-09-18T01:01:15.340720+0000","last_became_active":"2024-09-18T01:00:59.231756+0000","last_became_peered":"2024-09-18T01:00:59.231756+0000","last_unstale":"2024-09-18T01:01:15.340720+0000","last_undegraded":"2024-09-18T01:01:15.340720+0000","last_fullsized":"2024-09-18T01:01:15.340720+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:50:54.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"185'506","reported_seq":8883,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.341145+0000","last_change":"2024-09-18T01:00:44.150561+0000","last_active":"2024-09-18T01:01:15.341145+0000","last_peered":"2024-09-18T01:01:15.341145+0000","last_clean":"2024-09-18T01:01:15.341145+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T01:01:15.341145+0000","last_undegraded":"2024-09-18T01:01:15.341145+0000","last_fullsized":"2024-09-18T01:01:15.341145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:44.745709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036716330000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8918,"num_read_kb":44936,"num_write":410,"num_write_kb":13864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382260+0000","last_change":"2024-09-18T01:00:15.991240+0000","last_active":"2024-09-18T01:01:14.382260+0000","last_peered":"2024-09-18T01:01:14.382260+0000","last_clean":"2024-09-18T01:01:14.382260+0000","last_became_active":"2024-09-18T01:00:15.990930+0000","last_became_peered":"2024-09-18T01:00:15.990930+0000","last_unstale":"2024-09-18T01:01:14.382260+0000","last_undegraded":"2024-09-18T01:01:14.382260+0000","last_fullsized":"2024-09-18T01:01:14.382260+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:55.616429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"185'348","reported_seq":4491,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173966+0000","last_change":"2024-09-18T01:00:44.260854+0000","last_active":"2024-09-18T01:01:15.173966+0000","last_peered":"2024-09-18T01:01:15.173966+0000","last_clean":"2024-09-18T01:01:15.173966+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T01:01:15.173966+0000","last_undegraded":"2024-09-18T01:01:15.173966+0000","last_fullsized":"2024-09-18T01:01:15.173966+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:09.012851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019390989999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3644,"num_read_kb":27224,"num_write":289,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":399,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590664+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T01:01:15.590664+0000","last_peered":"2024-09-18T01:01:15.590664+0000","last_clean":"2024-09-18T01:01:15.590664+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T01:01:15.590664+0000","last_undegraded":"2024-09-18T01:01:15.590664+0000","last_fullsized":"2024-09-18T01:01:15.590664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"185'580","reported_seq":4494,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380809+0000","last_change":"2024-09-18T01:01:01.242483+0000","last_active":"2024-09-18T01:01:14.380809+0000","last_peered":"2024-09-18T01:01:14.380809+0000","last_clean":"2024-09-18T01:01:14.380809+0000","last_became_active":"2024-09-18T01:01:01.239957+0000","last_became_peered":"2024-09-18T01:01:01.239957+0000","last_unstale":"2024-09-18T01:01:14.380809+0000","last_undegraded":"2024-09-18T01:01:14.380809+0000","last_fullsized":"2024-09-18T01:01:14.380809+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:46.781823+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.024809728,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4830,"num_read_kb":33562,"num_write":800,"num_write_kb":24925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"186'946","reported_seq":6143,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941033+0000","last_change":"2024-09-18T01:01:03.958936+0000","last_active":"2024-09-18T01:01:14.941033+0000","last_peered":"2024-09-18T01:01:14.941033+0000","last_clean":"2024-09-18T01:01:14.941033+0000","last_became_active":"2024-09-18T01:01:03.955764+0000","last_became_peered":"2024-09-18T01:01:03.955764+0000","last_unstale":"2024-09-18T01:01:14.941033+0000","last_undegraded":"2024-09-18T01:01:14.941033+0000","last_fullsized":"2024-09-18T01:01:14.941033+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:25.024002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022401439999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7644,"num_read_kb":19178,"num_write":1076,"num_write_kb":33944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":99850853,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":56,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380143+0000","last_change":"2024-09-18T01:00:51.188692+0000","last_active":"2024-09-18T01:01:14.380143+0000","last_peered":"2024-09-18T01:01:14.380143+0000","last_clean":"2024-09-18T01:01:14.380143+0000","last_became_active":"2024-09-18T01:00:51.183713+0000","last_became_peered":"2024-09-18T01:00:51.183713+0000","last_unstale":"2024-09-18T01:01:14.380143+0000","last_undegraded":"2024-09-18T01:01:14.380143+0000","last_fullsized":"2024-09-18T01:01:14.380143+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:20:22.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"185'657","reported_seq":8382,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590875+0000","last_change":"2024-09-18T01:00:44.156905+0000","last_active":"2024-09-18T01:01:15.590875+0000","last_peered":"2024-09-18T01:01:15.590875+0000","last_clean":"2024-09-18T01:01:15.590875+0000","last_became_active":"2024-09-18T01:00:08.951992+0000","last_became_peered":"2024-09-18T01:00:08.951992+0000","last_unstale":"2024-09-18T01:01:15.590875+0000","last_undegraded":"2024-09-18T01:01:15.590875+0000","last_fullsized":"2024-09-18T01:01:15.590875+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:59.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030555019999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7663,"num_read_kb":29924,"num_write":748,"num_write_kb":19563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":15190015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381997+0000","last_change":"2024-09-18T01:00:04.976725+0000","last_active":"2024-09-18T01:01:14.381997+0000","last_peered":"2024-09-18T01:01:14.381997+0000","last_clean":"2024-09-18T01:01:14.381997+0000","last_became_active":"2024-09-18T01:00:04.976360+0000","last_became_peered":"2024-09-18T01:00:04.976360+0000","last_unstale":"2024-09-18T01:01:14.381997+0000","last_undegraded":"2024-09-18T01:01:14.381997+0000","last_fullsized":"2024-09-18T01:01:14.381997+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:11.159126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"184'548","reported_seq":9961,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173598+0000","last_change":"2024-09-18T01:01:01.957077+0000","last_active":"2024-09-18T01:01:15.173598+0000","last_peered":"2024-09-18T01:01:15.173598+0000","last_clean":"2024-09-18T01:01:15.173598+0000","last_became_active":"2024-09-18T01:01:01.953404+0000","last_became_peered":"2024-09-18T01:01:01.953404+0000","last_unstale":"2024-09-18T01:01:15.173598+0000","last_undegraded":"2024-09-18T01:01:15.173598+0000","last_fullsized":"2024-09-18T01:01:15.173598+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:03.050834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726180000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8919,"num_read_kb":48544,"num_write":559,"num_write_kb":18757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1056,"num_bytes_recovered":19034719,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379823+0000","last_change":"2024-09-18T01:00:15.994517+0000","last_active":"2024-09-18T01:01:14.379823+0000","last_peered":"2024-09-18T01:01:14.379823+0000","last_clean":"2024-09-18T01:01:14.379823+0000","last_became_active":"2024-09-18T01:00:15.994159+0000","last_became_peered":"2024-09-18T01:00:15.994159+0000","last_unstale":"2024-09-18T01:01:14.379823+0000","last_undegraded":"2024-09-18T01:01:14.379823+0000","last_fullsized":"2024-09-18T01:01:14.379823+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:06.775713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"186'722","reported_seq":5332,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340913+0000","last_change":"2024-09-18T01:01:01.952665+0000","last_active":"2024-09-18T01:01:15.340913+0000","last_peered":"2024-09-18T01:01:15.340913+0000","last_clean":"2024-09-18T01:01:15.340913+0000","last_became_active":"2024-09-18T01:01:01.948535+0000","last_became_peered":"2024-09-18T01:01:01.948535+0000","last_unstale":"2024-09-18T01:01:15.340913+0000","last_undegraded":"2024-09-18T01:01:15.340913+0000","last_fullsized":"2024-09-18T01:01:15.340913+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:39.959837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00139405,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5240,"num_read_kb":16439,"num_write":1146,"num_write_kb":52135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":12961092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":153,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380940+0000","last_change":"2024-09-18T01:00:58.232332+0000","last_active":"2024-09-18T01:01:14.380940+0000","last_peered":"2024-09-18T01:01:14.380940+0000","last_clean":"2024-09-18T01:01:14.380940+0000","last_became_active":"2024-09-18T01:00:58.229581+0000","last_became_peered":"2024-09-18T01:00:58.229581+0000","last_unstale":"2024-09-18T01:01:14.380940+0000","last_undegraded":"2024-09-18T01:01:14.380940+0000","last_fullsized":"2024-09-18T01:01:14.380940+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:05:38.818642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":551,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912328+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T01:01:14.912328+0000","last_peered":"2024-09-18T01:01:14.912328+0000","last_clean":"2024-09-18T01:01:14.912328+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T01:01:14.912328+0000","last_undegraded":"2024-09-18T01:01:14.912328+0000","last_fullsized":"2024-09-18T01:01:14.912328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"184'448","reported_seq":5879,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340789+0000","last_change":"2024-09-18T01:00:44.149901+0000","last_active":"2024-09-18T01:01:15.340789+0000","last_peered":"2024-09-18T01:01:15.340789+0000","last_clean":"2024-09-18T01:01:15.340789+0000","last_became_active":"2024-09-18T01:00:08.949384+0000","last_became_peered":"2024-09-18T01:00:08.949384+0000","last_unstale":"2024-09-18T01:01:15.340789+0000","last_undegraded":"2024-09-18T01:01:15.340789+0000","last_fullsized":"2024-09-18T01:01:15.340789+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:20.064612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033952990000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5722,"num_read_kb":51790,"num_write":458,"num_write_kb":39546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":32435091,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"185'408","reported_seq":3782,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379828+0000","last_change":"2024-09-18T01:00:44.145255+0000","last_active":"2024-09-18T01:01:14.379828+0000","last_peered":"2024-09-18T01:01:14.379828+0000","last_clean":"2024-09-18T01:01:14.379828+0000","last_became_active":"2024-09-18T01:00:18.997710+0000","last_became_peered":"2024-09-18T01:00:18.997710+0000","last_unstale":"2024-09-18T01:01:14.379828+0000","last_undegraded":"2024-09-18T01:01:14.379828+0000","last_fullsized":"2024-09-18T01:01:14.379828+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:36.454597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016382809999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3625,"num_read_kb":11553,"num_write":342,"num_write_kb":23969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":27148173,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"185'607","reported_seq":5628,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380626+0000","last_change":"2024-09-18T01:00:44.146011+0000","last_active":"2024-09-18T01:01:14.380626+0000","last_peered":"2024-09-18T01:01:14.380626+0000","last_clean":"2024-09-18T01:01:14.380626+0000","last_became_active":"2024-09-18T01:00:22.938344+0000","last_became_peered":"2024-09-18T01:00:22.938344+0000","last_unstale":"2024-09-18T01:01:14.380626+0000","last_undegraded":"2024-09-18T01:01:14.380626+0000","last_fullsized":"2024-09-18T01:01:14.380626+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:59:54.570595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021924589999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4754,"num_read_kb":46819,"num_write":596,"num_write_kb":17196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":6639875,"num_keys_recovered":85,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"185'515","reported_seq":5914,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382003+0000","last_change":"2024-09-18T01:00:44.262988+0000","last_active":"2024-09-18T01:01:14.382003+0000","last_peered":"2024-09-18T01:01:14.382003+0000","last_clean":"2024-09-18T01:01:14.382003+0000","last_became_active":"2024-09-18T01:00:10.954910+0000","last_became_peered":"2024-09-18T01:00:10.954910+0000","last_unstale":"2024-09-18T01:01:14.382003+0000","last_undegraded":"2024-09-18T01:01:14.382003+0000","last_fullsized":"2024-09-18T01:01:14.382003+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:45.069175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042875949999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5466,"num_read_kb":66445,"num_write":496,"num_write_kb":15031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15399944,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"185'566","reported_seq":7107,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941155+0000","last_change":"2024-09-18T01:01:02.956425+0000","last_active":"2024-09-18T01:01:14.941155+0000","last_peered":"2024-09-18T01:01:14.941155+0000","last_clean":"2024-09-18T01:01:14.941155+0000","last_became_active":"2024-09-18T01:01:02.953950+0000","last_became_peered":"2024-09-18T01:01:02.953950+0000","last_unstale":"2024-09-18T01:01:14.941155+0000","last_undegraded":"2024-09-18T01:01:14.941155+0000","last_fullsized":"2024-09-18T01:01:14.941155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:11.064860+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0026394880000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6401,"num_read_kb":47394,"num_write":633,"num_write_kb":22377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":560,"num_bytes_recovered":26783458,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"184'976","reported_seq":6804,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381748+0000","last_change":"2024-09-18T01:00:44.146438+0000","last_active":"2024-09-18T01:01:14.381748+0000","last_peered":"2024-09-18T01:01:14.381748+0000","last_clean":"2024-09-18T01:01:14.381748+0000","last_became_active":"2024-09-18T01:00:21.923279+0000","last_became_peered":"2024-09-18T01:00:21.923279+0000","last_unstale":"2024-09-18T01:01:14.381748+0000","last_undegraded":"2024-09-18T01:01:14.381748+0000","last_fullsized":"2024-09-18T01:01:14.381748+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:23.093467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026100659999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8636,"num_read_kb":40955,"num_write":1143,"num_write_kb":30743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":80305919,"num_keys_recovered":163,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"183'713","reported_seq":4745,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380694+0000","last_change":"2024-09-18T01:01:02.959668+0000","last_active":"2024-09-18T01:01:14.380694+0000","last_peered":"2024-09-18T01:01:14.380694+0000","last_clean":"2024-09-18T01:01:14.380694+0000","last_became_active":"2024-09-18T01:01:02.954337+0000","last_became_peered":"2024-09-18T01:01:02.954337+0000","last_unstale":"2024-09-18T01:01:14.380694+0000","last_undegraded":"2024-09-18T01:01:14.380694+0000","last_fullsized":"2024-09-18T01:01:14.380694+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:56:25.045514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0093305890000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4741,"num_read_kb":29100,"num_write":948,"num_write_kb":26466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":41587165,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"184'563","reported_seq":4380,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941359+0000","last_change":"2024-09-18T01:01:01.950143+0000","last_active":"2024-09-18T01:01:14.941359+0000","last_peered":"2024-09-18T01:01:14.941359+0000","last_clean":"2024-09-18T01:01:14.941359+0000","last_became_active":"2024-09-18T01:01:01.948267+0000","last_became_peered":"2024-09-18T01:01:01.948267+0000","last_unstale":"2024-09-18T01:01:14.941359+0000","last_undegraded":"2024-09-18T01:01:14.941359+0000","last_fullsized":"2024-09-18T01:01:14.941359+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:43:18.359056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024190230000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5311,"num_read_kb":28191,"num_write":598,"num_write_kb":9287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6320640,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"184'456","reported_seq":5302,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912243+0000","last_change":"2024-09-18T01:00:44.148986+0000","last_active":"2024-09-18T01:01:14.912243+0000","last_peered":"2024-09-18T01:01:14.912243+0000","last_clean":"2024-09-18T01:01:14.912243+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T01:01:14.912243+0000","last_undegraded":"2024-09-18T01:01:14.912243+0000","last_fullsized":"2024-09-18T01:01:14.912243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:48:20.609462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001443239,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":51584,"num_write":357,"num_write_kb":26802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"183'292","reported_seq":3444,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380096+0000","last_change":"2024-09-18T01:01:02.956466+0000","last_active":"2024-09-18T01:01:14.380096+0000","last_peered":"2024-09-18T01:01:14.380096+0000","last_clean":"2024-09-18T01:01:14.380096+0000","last_became_active":"2024-09-18T01:01:02.950361+0000","last_became_peered":"2024-09-18T01:01:02.950361+0000","last_unstale":"2024-09-18T01:01:14.380096+0000","last_undegraded":"2024-09-18T01:01:14.380096+0000","last_fullsized":"2024-09-18T01:01:14.380096+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:56:04.680353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033253200000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1897,"num_read_kb":13175,"num_write":246,"num_write_kb":14260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":15824068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"184'638","reported_seq":6909,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173643+0000","last_change":"2024-09-18T01:00:44.261997+0000","last_active":"2024-09-18T01:01:15.173643+0000","last_peered":"2024-09-18T01:01:15.173643+0000","last_clean":"2024-09-18T01:01:15.173643+0000","last_became_active":"2024-09-18T01:00:21.926814+0000","last_became_peered":"2024-09-18T01:00:21.926814+0000","last_unstale":"2024-09-18T01:01:15.173643+0000","last_undegraded":"2024-09-18T01:01:15.173643+0000","last_fullsized":"2024-09-18T01:01:15.173643+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:04.145432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002915932,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6954,"num_read_kb":17130,"num_write":754,"num_write_kb":11285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":32804395,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"185'521","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381219+0000","last_change":"2024-09-18T01:01:01.245238+0000","last_active":"2024-09-18T01:01:14.381219+0000","last_peered":"2024-09-18T01:01:14.381219+0000","last_clean":"2024-09-18T01:01:14.381219+0000","last_became_active":"2024-09-18T01:01:01.242541+0000","last_became_peered":"2024-09-18T01:01:01.242541+0000","last_unstale":"2024-09-18T01:01:14.381219+0000","last_undegraded":"2024-09-18T01:01:14.381219+0000","last_fullsized":"2024-09-18T01:01:14.381219+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:44.647014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027658140000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5114,"num_read_kb":26799,"num_write":585,"num_write_kb":11735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":9379374,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":482,"num_read":220594,"num_read_kb":1067291,"num_write":22261,"num_write_kb":688577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12013,"num_bytes_recovered":693846180,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19538,"ondisk_log_size":19538,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47592,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324248,"statfs":{"total":96636764160,"available":96588029952,"internally_reserved":0,"allocated":2572288,"data_stored":1237498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52344,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319496,"statfs":{"total":96636764160,"available":96583163904,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60064,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311776,"statfs":{"total":96636764160,"available":96575258624,"internally_reserved":0,"allocated":2568192,"data_stored":1237479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55920,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315920,"statfs":{"total":96636764160,"available":96579502080,"internally_reserved":0,"allocated":2584576,"data_stored":1243311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60080,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311760,"statfs":{"total":96636764160,"available":96575242240,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52344,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319496,"statfs":{"total":96636764160,"available":96583163904,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47600,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324240,"statfs":{"total":96636764160,"available":96588021760,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:01:18.165 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:01:18.165 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-18T01:01:18.467 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-18T01:01:18.476 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T01:01:18.477 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:01:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:01:18.766 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T01:01:18.782 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-18T01:01:17.946193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":482,"num_read":220710,"num_read_kb":1067391,"num_write":22422,"num_write_kb":690995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12021,"num_bytes_recovered":693846218,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19658,"ondisk_log_size":19658,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":427704,"kb_used_data":21312,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754547016,"statfs":{"total":773094113280,"available":772656144384,"internally_reserved":0,"allocated":21823488,"data_stored":11115522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":415944240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001933"},"pg_stats":[{"pgid":"3.1f","version":"186'621","reported_seq":4223,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590664+0000","last_change":"2024-09-18T01:00:44.155176+0000","last_active":"2024-09-18T01:01:15.590664+0000","last_peered":"2024-09-18T01:01:15.590664+0000","last_clean":"2024-09-18T01:01:15.590664+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T01:01:15.590664+0000","last_undegraded":"2024-09-18T01:01:15.590664+0000","last_fullsized":"2024-09-18T01:01:15.590664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:49.598474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001427074,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6437,"num_read_kb":23165,"num_write":614,"num_write_kb":11006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"186'646","reported_seq":5759,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590628+0000","last_change":"2024-09-18T01:01:01.248306+0000","last_active":"2024-09-18T01:01:15.590628+0000","last_peered":"2024-09-18T01:01:15.590628+0000","last_clean":"2024-09-18T01:01:15.590628+0000","last_became_active":"2024-09-18T01:01:01.245192+0000","last_became_peered":"2024-09-18T01:01:01.245192+0000","last_unstale":"2024-09-18T01:01:15.590628+0000","last_undegraded":"2024-09-18T01:01:15.590628+0000","last_fullsized":"2024-09-18T01:01:15.590628+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:50.236589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041620060000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4822,"num_read_kb":36392,"num_write":703,"num_write_kb":11559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":30825045,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"184'800","reported_seq":12971,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382040+0000","last_change":"2024-09-18T01:01:03.959251+0000","last_active":"2024-09-18T01:01:14.382040+0000","last_peered":"2024-09-18T01:01:14.382040+0000","last_clean":"2024-09-18T01:01:14.382040+0000","last_became_active":"2024-09-18T01:01:03.956061+0000","last_became_peered":"2024-09-18T01:01:03.956061+0000","last_unstale":"2024-09-18T01:01:14.382040+0000","last_undegraded":"2024-09-18T01:01:14.382040+0000","last_fullsized":"2024-09-18T01:01:14.382040+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:21:07.686876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034116569999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":13334,"num_read_kb":40113,"num_write":945,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":705,"num_bytes_recovered":22094834,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"186'663","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912412+0000","last_change":"2024-09-18T01:01:01.947287+0000","last_active":"2024-09-18T01:01:14.912412+0000","last_peered":"2024-09-18T01:01:14.912412+0000","last_clean":"2024-09-18T01:01:14.912412+0000","last_became_active":"2024-09-18T01:01:01.945747+0000","last_became_peered":"2024-09-18T01:01:01.945747+0000","last_unstale":"2024-09-18T01:01:14.912412+0000","last_undegraded":"2024-09-18T01:01:14.912412+0000","last_fullsized":"2024-09-18T01:01:14.912412+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:44:09.776110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002343262,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5693,"num_read_kb":28138,"num_write":736,"num_write_kb":11405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":5428683,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"186'1093","reported_seq":10876,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340934+0000","last_change":"2024-09-18T01:01:01.953195+0000","last_active":"2024-09-18T01:01:15.340934+0000","last_peered":"2024-09-18T01:01:15.340934+0000","last_clean":"2024-09-18T01:01:15.340934+0000","last_became_active":"2024-09-18T01:01:01.947957+0000","last_became_peered":"2024-09-18T01:01:01.947957+0000","last_unstale":"2024-09-18T01:01:15.340934+0000","last_undegraded":"2024-09-18T01:01:15.340934+0000","last_fullsized":"2024-09-18T01:01:15.340934+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:50:57.100165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003279527,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12686,"num_read_kb":40787,"num_write":1375,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":59340116,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"186'364","reported_seq":5144,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912518+0000","last_change":"2024-09-18T01:00:44.151068+0000","last_active":"2024-09-18T01:01:14.912518+0000","last_peered":"2024-09-18T01:01:14.912518+0000","last_clean":"2024-09-18T01:01:14.912518+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T01:01:14.912518+0000","last_undegraded":"2024-09-18T01:01:14.912518+0000","last_fullsized":"2024-09-18T01:01:14.912518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:24.750570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017298630000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5410,"num_read_kb":29581,"num_write":319,"num_write_kb":12768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"186'701","reported_seq":30888,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340790+0000","last_change":"2024-09-18T01:00:51.958851+0000","last_active":"2024-09-18T01:01:15.340790+0000","last_peered":"2024-09-18T01:01:15.340790+0000","last_clean":"2024-09-18T01:01:15.340790+0000","last_became_active":"2024-09-18T01:00:51.953781+0000","last_became_peered":"2024-09-18T01:00:51.953781+0000","last_unstale":"2024-09-18T01:01:15.340790+0000","last_undegraded":"2024-09-18T01:01:15.340790+0000","last_fullsized":"2024-09-18T01:01:15.340790+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:44.107525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001459178,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":30614,"num_read_kb":78260,"num_write":886,"num_write_kb":27079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":40291284,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"186'656","reported_seq":4017,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.591127+0000","last_change":"2024-09-18T01:00:44.157985+0000","last_active":"2024-09-18T01:01:15.591127+0000","last_peered":"2024-09-18T01:01:15.591127+0000","last_clean":"2024-09-18T01:01:15.591127+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T01:01:15.591127+0000","last_undegraded":"2024-09-18T01:01:15.591127+0000","last_fullsized":"2024-09-18T01:01:15.591127+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:27.075227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039694869999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4869,"num_read_kb":13814,"num_write":877,"num_write_kb":42194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"185'676","reported_seq":7355,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590783+0000","last_change":"2024-09-18T01:00:44.156705+0000","last_active":"2024-09-18T01:01:15.590783+0000","last_peered":"2024-09-18T01:01:15.590783+0000","last_clean":"2024-09-18T01:01:15.590783+0000","last_became_active":"2024-09-18T01:00:20.017711+0000","last_became_peered":"2024-09-18T01:00:20.017711+0000","last_unstale":"2024-09-18T01:01:15.590783+0000","last_undegraded":"2024-09-18T01:01:15.590783+0000","last_fullsized":"2024-09-18T01:01:15.590783+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:49.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028584439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7284,"num_read_kb":46982,"num_write":773,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":13346337,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"186'641","reported_seq":8980,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380983+0000","last_change":"2024-09-18T01:01:01.955368+0000","last_active":"2024-09-18T01:01:14.380983+0000","last_peered":"2024-09-18T01:01:14.380983+0000","last_clean":"2024-09-18T01:01:14.380983+0000","last_became_active":"2024-09-18T01:01:01.952512+0000","last_became_peered":"2024-09-18T01:01:01.952512+0000","last_unstale":"2024-09-18T01:01:14.380983+0000","last_undegraded":"2024-09-18T01:01:14.380983+0000","last_fullsized":"2024-09-18T01:01:14.380983+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:36:14.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887849999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8507,"num_read_kb":37252,"num_write":699,"num_write_kb":22150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":946,"num_bytes_recovered":37601010,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"184'519","reported_seq":3007,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941255+0000","last_change":"2024-09-18T01:01:02.959624+0000","last_active":"2024-09-18T01:01:14.941255+0000","last_peered":"2024-09-18T01:01:14.941255+0000","last_clean":"2024-09-18T01:01:14.941255+0000","last_became_active":"2024-09-18T01:01:02.956655+0000","last_became_peered":"2024-09-18T01:01:02.956655+0000","last_unstale":"2024-09-18T01:01:14.941255+0000","last_undegraded":"2024-09-18T01:01:14.941255+0000","last_fullsized":"2024-09-18T01:01:14.941255+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:53:56.284343+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010838409,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2322,"num_read_kb":15300,"num_write":809,"num_write_kb":48070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":22291504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"181'371","reported_seq":3210,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381490+0000","last_change":"2024-09-18T01:01:01.948935+0000","last_active":"2024-09-18T01:01:14.381490+0000","last_peered":"2024-09-18T01:01:14.381490+0000","last_clean":"2024-09-18T01:01:14.381490+0000","last_became_active":"2024-09-18T01:01:01.947158+0000","last_became_peered":"2024-09-18T01:01:01.947158+0000","last_unstale":"2024-09-18T01:01:14.381490+0000","last_undegraded":"2024-09-18T01:01:14.381490+0000","last_fullsized":"2024-09-18T01:01:14.381490+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:05.169088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013361650000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2452,"num_read_kb":3816,"num_write":339,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":3474495,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":127,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382692+0000","last_change":"2024-09-18T01:00:59.238970+0000","last_active":"2024-09-18T01:01:14.382692+0000","last_peered":"2024-09-18T01:01:14.382692+0000","last_clean":"2024-09-18T01:01:14.382692+0000","last_became_active":"2024-09-18T01:00:59.238106+0000","last_became_peered":"2024-09-18T01:00:59.238106+0000","last_unstale":"2024-09-18T01:01:14.382692+0000","last_undegraded":"2024-09-18T01:01:14.382692+0000","last_fullsized":"2024-09-18T01:01:14.382692+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:15:05.506655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"186'777","reported_seq":4883,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.341002+0000","last_change":"2024-09-18T01:00:44.148713+0000","last_active":"2024-09-18T01:01:15.341002+0000","last_peered":"2024-09-18T01:01:15.341002+0000","last_clean":"2024-09-18T01:01:15.341002+0000","last_became_active":"2024-09-18T01:00:11.936218+0000","last_became_peered":"2024-09-18T01:00:11.936218+0000","last_unstale":"2024-09-18T01:01:15.341002+0000","last_undegraded":"2024-09-18T01:01:15.341002+0000","last_fullsized":"2024-09-18T01:01:15.341002+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:24.939297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002351681,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5488,"num_read_kb":22949,"num_write":1002,"num_write_kb":4650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":2805113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":412,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340720+0000","last_change":"2024-09-18T01:00:59.233973+0000","last_active":"2024-09-18T01:01:15.340720+0000","last_peered":"2024-09-18T01:01:15.340720+0000","last_clean":"2024-09-18T01:01:15.340720+0000","last_became_active":"2024-09-18T01:00:59.231756+0000","last_became_peered":"2024-09-18T01:00:59.231756+0000","last_unstale":"2024-09-18T01:01:15.340720+0000","last_undegraded":"2024-09-18T01:01:15.340720+0000","last_fullsized":"2024-09-18T01:01:15.340720+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:50:54.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"185'506","reported_seq":8883,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.341145+0000","last_change":"2024-09-18T01:00:44.150561+0000","last_active":"2024-09-18T01:01:15.341145+0000","last_peered":"2024-09-18T01:01:15.341145+0000","last_clean":"2024-09-18T01:01:15.341145+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T01:01:15.341145+0000","last_undegraded":"2024-09-18T01:01:15.341145+0000","last_fullsized":"2024-09-18T01:01:15.341145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:44.745709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036716330000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8918,"num_read_kb":44936,"num_write":410,"num_write_kb":13864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382260+0000","last_change":"2024-09-18T01:00:15.991240+0000","last_active":"2024-09-18T01:01:14.382260+0000","last_peered":"2024-09-18T01:01:14.382260+0000","last_clean":"2024-09-18T01:01:14.382260+0000","last_became_active":"2024-09-18T01:00:15.990930+0000","last_became_peered":"2024-09-18T01:00:15.990930+0000","last_unstale":"2024-09-18T01:01:14.382260+0000","last_undegraded":"2024-09-18T01:01:14.382260+0000","last_fullsized":"2024-09-18T01:01:14.382260+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:55.616429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"185'348","reported_seq":4491,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173966+0000","last_change":"2024-09-18T01:00:44.260854+0000","last_active":"2024-09-18T01:01:15.173966+0000","last_peered":"2024-09-18T01:01:15.173966+0000","last_clean":"2024-09-18T01:01:15.173966+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T01:01:15.173966+0000","last_undegraded":"2024-09-18T01:01:15.173966+0000","last_fullsized":"2024-09-18T01:01:15.173966+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:09.012851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019390989999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3644,"num_read_kb":27224,"num_write":289,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":399,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590664+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T01:01:15.590664+0000","last_peered":"2024-09-18T01:01:15.590664+0000","last_clean":"2024-09-18T01:01:15.590664+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T01:01:15.590664+0000","last_undegraded":"2024-09-18T01:01:15.590664+0000","last_fullsized":"2024-09-18T01:01:15.590664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"185'580","reported_seq":4494,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380809+0000","last_change":"2024-09-18T01:01:01.242483+0000","last_active":"2024-09-18T01:01:14.380809+0000","last_peered":"2024-09-18T01:01:14.380809+0000","last_clean":"2024-09-18T01:01:14.380809+0000","last_became_active":"2024-09-18T01:01:01.239957+0000","last_became_peered":"2024-09-18T01:01:01.239957+0000","last_unstale":"2024-09-18T01:01:14.380809+0000","last_undegraded":"2024-09-18T01:01:14.380809+0000","last_fullsized":"2024-09-18T01:01:14.380809+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:46.781823+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.024809728,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4830,"num_read_kb":33562,"num_write":800,"num_write_kb":24925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"186'946","reported_seq":6143,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941033+0000","last_change":"2024-09-18T01:01:03.958936+0000","last_active":"2024-09-18T01:01:14.941033+0000","last_peered":"2024-09-18T01:01:14.941033+0000","last_clean":"2024-09-18T01:01:14.941033+0000","last_became_active":"2024-09-18T01:01:03.955764+0000","last_became_peered":"2024-09-18T01:01:03.955764+0000","last_unstale":"2024-09-18T01:01:14.941033+0000","last_undegraded":"2024-09-18T01:01:14.941033+0000","last_fullsized":"2024-09-18T01:01:14.941033+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:25.024002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022401439999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7644,"num_read_kb":19178,"num_write":1076,"num_write_kb":33944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":99850853,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":56,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380143+0000","last_change":"2024-09-18T01:00:51.188692+0000","last_active":"2024-09-18T01:01:14.380143+0000","last_peered":"2024-09-18T01:01:14.380143+0000","last_clean":"2024-09-18T01:01:14.380143+0000","last_became_active":"2024-09-18T01:00:51.183713+0000","last_became_peered":"2024-09-18T01:00:51.183713+0000","last_unstale":"2024-09-18T01:01:14.380143+0000","last_undegraded":"2024-09-18T01:01:14.380143+0000","last_fullsized":"2024-09-18T01:01:14.380143+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:20:22.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"185'657","reported_seq":8382,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590875+0000","last_change":"2024-09-18T01:00:44.156905+0000","last_active":"2024-09-18T01:01:15.590875+0000","last_peered":"2024-09-18T01:01:15.590875+0000","last_clean":"2024-09-18T01:01:15.590875+0000","last_became_active":"2024-09-18T01:00:08.951992+0000","last_became_peered":"2024-09-18T01:00:08.951992+0000","last_unstale":"2024-09-18T01:01:15.590875+0000","last_undegraded":"2024-09-18T01:01:15.590875+0000","last_fullsized":"2024-09-18T01:01:15.590875+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:59.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030555019999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7663,"num_read_kb":29924,"num_write":748,"num_write_kb":19563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":15190015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381997+0000","last_change":"2024-09-18T01:00:04.976725+0000","last_active":"2024-09-18T01:01:14.381997+0000","last_peered":"2024-09-18T01:01:14.381997+0000","last_clean":"2024-09-18T01:01:14.381997+0000","last_became_active":"2024-09-18T01:00:04.976360+0000","last_became_peered":"2024-09-18T01:00:04.976360+0000","last_unstale":"2024-09-18T01:01:14.381997+0000","last_undegraded":"2024-09-18T01:01:14.381997+0000","last_fullsized":"2024-09-18T01:01:14.381997+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:11.159126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"184'548","reported_seq":9961,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173598+0000","last_change":"2024-09-18T01:01:01.957077+0000","last_active":"2024-09-18T01:01:15.173598+0000","last_peered":"2024-09-18T01:01:15.173598+0000","last_clean":"2024-09-18T01:01:15.173598+0000","last_became_active":"2024-09-18T01:01:01.953404+0000","last_became_peered":"2024-09-18T01:01:01.953404+0000","last_unstale":"2024-09-18T01:01:15.173598+0000","last_undegraded":"2024-09-18T01:01:15.173598+0000","last_fullsized":"2024-09-18T01:01:15.173598+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:03.050834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726180000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8919,"num_read_kb":48544,"num_write":559,"num_write_kb":18757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1056,"num_bytes_recovered":19034719,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379823+0000","last_change":"2024-09-18T01:00:15.994517+0000","last_active":"2024-09-18T01:01:14.379823+0000","last_peered":"2024-09-18T01:01:14.379823+0000","last_clean":"2024-09-18T01:01:14.379823+0000","last_became_active":"2024-09-18T01:00:15.994159+0000","last_became_peered":"2024-09-18T01:00:15.994159+0000","last_unstale":"2024-09-18T01:01:14.379823+0000","last_undegraded":"2024-09-18T01:01:14.379823+0000","last_fullsized":"2024-09-18T01:01:14.379823+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:06.775713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"186'722","reported_seq":5332,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340913+0000","last_change":"2024-09-18T01:01:01.952665+0000","last_active":"2024-09-18T01:01:15.340913+0000","last_peered":"2024-09-18T01:01:15.340913+0000","last_clean":"2024-09-18T01:01:15.340913+0000","last_became_active":"2024-09-18T01:01:01.948535+0000","last_became_peered":"2024-09-18T01:01:01.948535+0000","last_unstale":"2024-09-18T01:01:15.340913+0000","last_undegraded":"2024-09-18T01:01:15.340913+0000","last_fullsized":"2024-09-18T01:01:15.340913+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:39.959837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00139405,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5240,"num_read_kb":16439,"num_write":1146,"num_write_kb":52135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":12961092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":153,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380940+0000","last_change":"2024-09-18T01:00:58.232332+0000","last_active":"2024-09-18T01:01:14.380940+0000","last_peered":"2024-09-18T01:01:14.380940+0000","last_clean":"2024-09-18T01:01:14.380940+0000","last_became_active":"2024-09-18T01:00:58.229581+0000","last_became_peered":"2024-09-18T01:00:58.229581+0000","last_unstale":"2024-09-18T01:01:14.380940+0000","last_undegraded":"2024-09-18T01:01:14.380940+0000","last_fullsized":"2024-09-18T01:01:14.380940+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:05:38.818642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":551,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912328+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T01:01:14.912328+0000","last_peered":"2024-09-18T01:01:14.912328+0000","last_clean":"2024-09-18T01:01:14.912328+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T01:01:14.912328+0000","last_undegraded":"2024-09-18T01:01:14.912328+0000","last_fullsized":"2024-09-18T01:01:14.912328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"184'448","reported_seq":5879,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340789+0000","last_change":"2024-09-18T01:00:44.149901+0000","last_active":"2024-09-18T01:01:15.340789+0000","last_peered":"2024-09-18T01:01:15.340789+0000","last_clean":"2024-09-18T01:01:15.340789+0000","last_became_active":"2024-09-18T01:00:08.949384+0000","last_became_peered":"2024-09-18T01:00:08.949384+0000","last_unstale":"2024-09-18T01:01:15.340789+0000","last_undegraded":"2024-09-18T01:01:15.340789+0000","last_fullsized":"2024-09-18T01:01:15.340789+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:20.064612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033952990000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5722,"num_read_kb":51790,"num_write":458,"num_write_kb":39546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":32435091,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"185'408","reported_seq":3782,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379828+0000","last_change":"2024-09-18T01:00:44.145255+0000","last_active":"2024-09-18T01:01:14.379828+0000","last_peered":"2024-09-18T01:01:14.379828+0000","last_clean":"2024-09-18T01:01:14.379828+0000","last_became_active":"2024-09-18T01:00:18.997710+0000","last_became_peered":"2024-09-18T01:00:18.997710+0000","last_unstale":"2024-09-18T01:01:14.379828+0000","last_undegraded":"2024-09-18T01:01:14.379828+0000","last_fullsized":"2024-09-18T01:01:14.379828+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:36.454597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016382809999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3625,"num_read_kb":11553,"num_write":342,"num_write_kb":23969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":27148173,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"185'607","reported_seq":5628,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380626+0000","last_change":"2024-09-18T01:00:44.146011+0000","last_active":"2024-09-18T01:01:14.380626+0000","last_peered":"2024-09-18T01:01:14.380626+0000","last_clean":"2024-09-18T01:01:14.380626+0000","last_became_active":"2024-09-18T01:00:22.938344+0000","last_became_peered":"2024-09-18T01:00:22.938344+0000","last_unstale":"2024-09-18T01:01:14.380626+0000","last_undegraded":"2024-09-18T01:01:14.380626+0000","last_fullsized":"2024-09-18T01:01:14.380626+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:59:54.570595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021924589999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4754,"num_read_kb":46819,"num_write":596,"num_write_kb":17196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":6639875,"num_keys_recovered":85,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"185'515","reported_seq":5914,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382003+0000","last_change":"2024-09-18T01:00:44.262988+0000","last_active":"2024-09-18T01:01:14.382003+0000","last_peered":"2024-09-18T01:01:14.382003+0000","last_clean":"2024-09-18T01:01:14.382003+0000","last_became_active":"2024-09-18T01:00:10.954910+0000","last_became_peered":"2024-09-18T01:00:10.954910+0000","last_unstale":"2024-09-18T01:01:14.382003+0000","last_undegraded":"2024-09-18T01:01:14.382003+0000","last_fullsized":"2024-09-18T01:01:14.382003+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:45.069175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042875949999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5466,"num_read_kb":66445,"num_write":496,"num_write_kb":15031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15399944,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"185'566","reported_seq":7107,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941155+0000","last_change":"2024-09-18T01:01:02.956425+0000","last_active":"2024-09-18T01:01:14.941155+0000","last_peered":"2024-09-18T01:01:14.941155+0000","last_clean":"2024-09-18T01:01:14.941155+0000","last_became_active":"2024-09-18T01:01:02.953950+0000","last_became_peered":"2024-09-18T01:01:02.953950+0000","last_unstale":"2024-09-18T01:01:14.941155+0000","last_undegraded":"2024-09-18T01:01:14.941155+0000","last_fullsized":"2024-09-18T01:01:14.941155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:11.064860+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0026394880000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6401,"num_read_kb":47394,"num_write":633,"num_write_kb":22377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":560,"num_bytes_recovered":26783458,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"184'976","reported_seq":6804,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381748+0000","last_change":"2024-09-18T01:00:44.146438+0000","last_active":"2024-09-18T01:01:14.381748+0000","last_peered":"2024-09-18T01:01:14.381748+0000","last_clean":"2024-09-18T01:01:14.381748+0000","last_became_active":"2024-09-18T01:00:21.923279+0000","last_became_peered":"2024-09-18T01:00:21.923279+0000","last_unstale":"2024-09-18T01:01:14.381748+0000","last_undegraded":"2024-09-18T01:01:14.381748+0000","last_fullsized":"2024-09-18T01:01:14.381748+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:23.093467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026100659999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8636,"num_read_kb":40955,"num_write":1143,"num_write_kb":30743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":80305919,"num_keys_recovered":163,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"183'713","reported_seq":4745,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380694+0000","last_change":"2024-09-18T01:01:02.959668+0000","last_active":"2024-09-18T01:01:14.380694+0000","last_peered":"2024-09-18T01:01:14.380694+0000","last_clean":"2024-09-18T01:01:14.380694+0000","last_became_active":"2024-09-18T01:01:02.954337+0000","last_became_peered":"2024-09-18T01:01:02.954337+0000","last_unstale":"2024-09-18T01:01:14.380694+0000","last_undegraded":"2024-09-18T01:01:14.380694+0000","last_fullsized":"2024-09-18T01:01:14.380694+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:56:25.045514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0093305890000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4741,"num_read_kb":29100,"num_write":948,"num_write_kb":26466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":41587165,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"184'563","reported_seq":4380,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941359+0000","last_change":"2024-09-18T01:01:01.950143+0000","last_active":"2024-09-18T01:01:14.941359+0000","last_peered":"2024-09-18T01:01:14.941359+0000","last_clean":"2024-09-18T01:01:14.941359+0000","last_became_active":"2024-09-18T01:01:01.948267+0000","last_became_peered":"2024-09-18T01:01:01.948267+0000","last_unstale":"2024-09-18T01:01:14.941359+0000","last_undegraded":"2024-09-18T01:01:14.941359+0000","last_fullsized":"2024-09-18T01:01:14.941359+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:43:18.359056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024190230000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5311,"num_read_kb":28191,"num_write":598,"num_write_kb":9287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6320640,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"184'456","reported_seq":5302,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912243+0000","last_change":"2024-09-18T01:00:44.148986+0000","last_active":"2024-09-18T01:01:14.912243+0000","last_peered":"2024-09-18T01:01:14.912243+0000","last_clean":"2024-09-18T01:01:14.912243+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T01:01:14.912243+0000","last_undegraded":"2024-09-18T01:01:14.912243+0000","last_fullsized":"2024-09-18T01:01:14.912243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:48:20.609462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001443239,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":51584,"num_write":357,"num_write_kb":26802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"183'292","reported_seq":3444,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380096+0000","last_change":"2024-09-18T01:01:02.956466+0000","last_active":"2024-09-18T01:01:14.380096+0000","last_peered":"2024-09-18T01:01:14.380096+0000","last_clean":"2024-09-18T01:01:14.380096+0000","last_became_active":"2024-09-18T01:01:02.950361+0000","last_became_peered":"2024-09-18T01:01:02.950361+0000","last_unstale":"2024-09-18T01:01:14.380096+0000","last_undegraded":"2024-09-18T01:01:14.380096+0000","last_fullsized":"2024-09-18T01:01:14.380096+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:56:04.680353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033253200000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1897,"num_read_kb":13175,"num_write":246,"num_write_kb":14260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":15824068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"184'638","reported_seq":6909,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173643+0000","last_change":"2024-09-18T01:00:44.261997+0000","last_active":"2024-09-18T01:01:15.173643+0000","last_peered":"2024-09-18T01:01:15.173643+0000","last_clean":"2024-09-18T01:01:15.173643+0000","last_became_active":"2024-09-18T01:00:21.926814+0000","last_became_peered":"2024-09-18T01:00:21.926814+0000","last_unstale":"2024-09-18T01:01:15.173643+0000","last_undegraded":"2024-09-18T01:01:15.173643+0000","last_fullsized":"2024-09-18T01:01:15.173643+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:04.145432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002915932,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6954,"num_read_kb":17130,"num_write":754,"num_write_kb":11285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":32804395,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"185'521","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381219+0000","last_change":"2024-09-18T01:01:01.245238+0000","last_active":"2024-09-18T01:01:14.381219+0000","last_peered":"2024-09-18T01:01:14.381219+0000","last_clean":"2024-09-18T01:01:14.381219+0000","last_became_active":"2024-09-18T01:01:01.242541+0000","last_became_peered":"2024-09-18T01:01:01.242541+0000","last_unstale":"2024-09-18T01:01:14.381219+0000","last_undegraded":"2024-09-18T01:01:14.381219+0000","last_fullsized":"2024-09-18T01:01:14.381219+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:44.647014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027658140000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5114,"num_read_kb":26799,"num_write":585,"num_write_kb":11735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":9379374,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":482,"num_read":220594,"num_read_kb":1067291,"num_write":22261,"num_write_kb":688577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12013,"num_bytes_recovered":693846180,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19538,"ondisk_log_size":19538,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47592,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324248,"statfs":{"total":96636764160,"available":96588029952,"internally_reserved":0,"allocated":2572288,"data_stored":1237498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52344,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319496,"statfs":{"total":96636764160,"available":96583163904,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60064,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311776,"statfs":{"total":96636764160,"available":96575258624,"internally_reserved":0,"allocated":2568192,"data_stored":1237479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55920,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315920,"statfs":{"total":96636764160,"available":96579502080,"internally_reserved":0,"allocated":2584576,"data_stored":1243311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60080,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311760,"statfs":{"total":96636764160,"available":96575242240,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52344,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319496,"statfs":{"total":96636764160,"available":96583163904,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47600,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324240,"statfs":{"total":96636764160,"available":96588021760,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:01:18.785 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:01:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:01:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T01:01:19.087 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-18T01:01:17.946193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":482,"num_read":220710,"num_read_kb":1067391,"num_write":22422,"num_write_kb":690995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12021,"num_bytes_recovered":693846218,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19658,"ondisk_log_size":19658,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":427704,"kb_used_data":21312,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754547016,"statfs":{"total":773094113280,"available":772656144384,"internally_reserved":0,"allocated":21823488,"data_stored":11115522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":415944240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001933"},"pg_stats":[{"pgid":"3.1f","version":"186'621","reported_seq":4223,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590664+0000","last_change":"2024-09-18T01:00:44.155176+0000","last_active":"2024-09-18T01:01:15.590664+0000","last_peered":"2024-09-18T01:01:15.590664+0000","last_clean":"2024-09-18T01:01:15.590664+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T01:01:15.590664+0000","last_undegraded":"2024-09-18T01:01:15.590664+0000","last_fullsized":"2024-09-18T01:01:15.590664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:49.598474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001427074,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6437,"num_read_kb":23165,"num_write":614,"num_write_kb":11006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"186'646","reported_seq":5759,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590628+0000","last_change":"2024-09-18T01:01:01.248306+0000","last_active":"2024-09-18T01:01:15.590628+0000","last_peered":"2024-09-18T01:01:15.590628+0000","last_clean":"2024-09-18T01:01:15.590628+0000","last_became_active":"2024-09-18T01:01:01.245192+0000","last_became_peered":"2024-09-18T01:01:01.245192+0000","last_unstale":"2024-09-18T01:01:15.590628+0000","last_undegraded":"2024-09-18T01:01:15.590628+0000","last_fullsized":"2024-09-18T01:01:15.590628+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:50.236589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041620060000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4822,"num_read_kb":36392,"num_write":703,"num_write_kb":11559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":30825045,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"184'800","reported_seq":12971,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382040+0000","last_change":"2024-09-18T01:01:03.959251+0000","last_active":"2024-09-18T01:01:14.382040+0000","last_peered":"2024-09-18T01:01:14.382040+0000","last_clean":"2024-09-18T01:01:14.382040+0000","last_became_active":"2024-09-18T01:01:03.956061+0000","last_became_peered":"2024-09-18T01:01:03.956061+0000","last_unstale":"2024-09-18T01:01:14.382040+0000","last_undegraded":"2024-09-18T01:01:14.382040+0000","last_fullsized":"2024-09-18T01:01:14.382040+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:21:07.686876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034116569999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":13334,"num_read_kb":40113,"num_write":945,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":705,"num_bytes_recovered":22094834,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"186'663","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912412+0000","last_change":"2024-09-18T01:01:01.947287+0000","last_active":"2024-09-18T01:01:14.912412+0000","last_peered":"2024-09-18T01:01:14.912412+0000","last_clean":"2024-09-18T01:01:14.912412+0000","last_became_active":"2024-09-18T01:01:01.945747+0000","last_became_peered":"2024-09-18T01:01:01.945747+0000","last_unstale":"2024-09-18T01:01:14.912412+0000","last_undegraded":"2024-09-18T01:01:14.912412+0000","last_fullsized":"2024-09-18T01:01:14.912412+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:44:09.776110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002343262,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5693,"num_read_kb":28138,"num_write":736,"num_write_kb":11405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":5428683,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"186'1093","reported_seq":10876,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340934+0000","last_change":"2024-09-18T01:01:01.953195+0000","last_active":"2024-09-18T01:01:15.340934+0000","last_peered":"2024-09-18T01:01:15.340934+0000","last_clean":"2024-09-18T01:01:15.340934+0000","last_became_active":"2024-09-18T01:01:01.947957+0000","last_became_peered":"2024-09-18T01:01:01.947957+0000","last_unstale":"2024-09-18T01:01:15.340934+0000","last_undegraded":"2024-09-18T01:01:15.340934+0000","last_fullsized":"2024-09-18T01:01:15.340934+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:50:57.100165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003279527,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12686,"num_read_kb":40787,"num_write":1375,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":59340116,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"186'364","reported_seq":5144,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912518+0000","last_change":"2024-09-18T01:00:44.151068+0000","last_active":"2024-09-18T01:01:14.912518+0000","last_peered":"2024-09-18T01:01:14.912518+0000","last_clean":"2024-09-18T01:01:14.912518+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T01:01:14.912518+0000","last_undegraded":"2024-09-18T01:01:14.912518+0000","last_fullsized":"2024-09-18T01:01:14.912518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:24.750570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017298630000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5410,"num_read_kb":29581,"num_write":319,"num_write_kb":12768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"186'701","reported_seq":30888,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340790+0000","last_change":"2024-09-18T01:00:51.958851+0000","last_active":"2024-09-18T01:01:15.340790+0000","last_peered":"2024-09-18T01:01:15.340790+0000","last_clean":"2024-09-18T01:01:15.340790+0000","last_became_active":"2024-09-18T01:00:51.953781+0000","last_became_peered":"2024-09-18T01:00:51.953781+0000","last_unstale":"2024-09-18T01:01:15.340790+0000","last_undegraded":"2024-09-18T01:01:15.340790+0000","last_fullsized":"2024-09-18T01:01:15.340790+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:44.107525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001459178,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":30614,"num_read_kb":78260,"num_write":886,"num_write_kb":27079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":40291284,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"186'656","reported_seq":4017,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.591127+0000","last_change":"2024-09-18T01:00:44.157985+0000","last_active":"2024-09-18T01:01:15.591127+0000","last_peered":"2024-09-18T01:01:15.591127+0000","last_clean":"2024-09-18T01:01:15.591127+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T01:01:15.591127+0000","last_undegraded":"2024-09-18T01:01:15.591127+0000","last_fullsized":"2024-09-18T01:01:15.591127+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:27.075227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039694869999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4869,"num_read_kb":13814,"num_write":877,"num_write_kb":42194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"185'676","reported_seq":7355,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590783+0000","last_change":"2024-09-18T01:00:44.156705+0000","last_active":"2024-09-18T01:01:15.590783+0000","last_peered":"2024-09-18T01:01:15.590783+0000","last_clean":"2024-09-18T01:01:15.590783+0000","last_became_active":"2024-09-18T01:00:20.017711+0000","last_became_peered":"2024-09-18T01:00:20.017711+0000","last_unstale":"2024-09-18T01:01:15.590783+0000","last_undegraded":"2024-09-18T01:01:15.590783+0000","last_fullsized":"2024-09-18T01:01:15.590783+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:49.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028584439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7284,"num_read_kb":46982,"num_write":773,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":13346337,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"186'641","reported_seq":8980,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380983+0000","last_change":"2024-09-18T01:01:01.955368+0000","last_active":"2024-09-18T01:01:14.380983+0000","last_peered":"2024-09-18T01:01:14.380983+0000","last_clean":"2024-09-18T01:01:14.380983+0000","last_became_active":"2024-09-18T01:01:01.952512+0000","last_became_peered":"2024-09-18T01:01:01.952512+0000","last_unstale":"2024-09-18T01:01:14.380983+0000","last_undegraded":"2024-09-18T01:01:14.380983+0000","last_fullsized":"2024-09-18T01:01:14.380983+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:36:14.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887849999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8507,"num_read_kb":37252,"num_write":699,"num_write_kb":22150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":946,"num_bytes_recovered":37601010,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"184'519","reported_seq":3007,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941255+0000","last_change":"2024-09-18T01:01:02.959624+0000","last_active":"2024-09-18T01:01:14.941255+0000","last_peered":"2024-09-18T01:01:14.941255+0000","last_clean":"2024-09-18T01:01:14.941255+0000","last_became_active":"2024-09-18T01:01:02.956655+0000","last_became_peered":"2024-09-18T01:01:02.956655+0000","last_unstale":"2024-09-18T01:01:14.941255+0000","last_undegraded":"2024-09-18T01:01:14.941255+0000","last_fullsized":"2024-09-18T01:01:14.941255+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:53:56.284343+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010838409,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2322,"num_read_kb":15300,"num_write":809,"num_write_kb":48070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":22291504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"181'371","reported_seq":3210,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381490+0000","last_change":"2024-09-18T01:01:01.948935+0000","last_active":"2024-09-18T01:01:14.381490+0000","last_peered":"2024-09-18T01:01:14.381490+0000","last_clean":"2024-09-18T01:01:14.381490+0000","last_became_active":"2024-09-18T01:01:01.947158+0000","last_became_peered":"2024-09-18T01:01:01.947158+0000","last_unstale":"2024-09-18T01:01:14.381490+0000","last_undegraded":"2024-09-18T01:01:14.381490+0000","last_fullsized":"2024-09-18T01:01:14.381490+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:05.169088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013361650000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2452,"num_read_kb":3816,"num_write":339,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":3474495,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":127,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382692+0000","last_change":"2024-09-18T01:00:59.238970+0000","last_active":"2024-09-18T01:01:14.382692+0000","last_peered":"2024-09-18T01:01:14.382692+0000","last_clean":"2024-09-18T01:01:14.382692+0000","last_became_active":"2024-09-18T01:00:59.238106+0000","last_became_peered":"2024-09-18T01:00:59.238106+0000","last_unstale":"2024-09-18T01:01:14.382692+0000","last_undegraded":"2024-09-18T01:01:14.382692+0000","last_fullsized":"2024-09-18T01:01:14.382692+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:15:05.506655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"186'777","reported_seq":4883,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.341002+0000","last_change":"2024-09-18T01:00:44.148713+0000","last_active":"2024-09-18T01:01:15.341002+0000","last_peered":"2024-09-18T01:01:15.341002+0000","last_clean":"2024-09-18T01:01:15.341002+0000","last_became_active":"2024-09-18T01:00:11.936218+0000","last_became_peered":"2024-09-18T01:00:11.936218+0000","last_unstale":"2024-09-18T01:01:15.341002+0000","last_undegraded":"2024-09-18T01:01:15.341002+0000","last_fullsized":"2024-09-18T01:01:15.341002+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:24.939297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002351681,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5488,"num_read_kb":22949,"num_write":1002,"num_write_kb":4650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":2805113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":412,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340720+0000","last_change":"2024-09-18T01:00:59.233973+0000","last_active":"2024-09-18T01:01:15.340720+0000","last_peered":"2024-09-18T01:01:15.340720+0000","last_clean":"2024-09-18T01:01:15.340720+0000","last_became_active":"2024-09-18T01:00:59.231756+0000","last_became_peered":"2024-09-18T01:00:59.231756+0000","last_unstale":"2024-09-18T01:01:15.340720+0000","last_undegraded":"2024-09-18T01:01:15.340720+0000","last_fullsized":"2024-09-18T01:01:15.340720+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:50:54.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"185'506","reported_seq":8883,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.341145+0000","last_change":"2024-09-18T01:00:44.150561+0000","last_active":"2024-09-18T01:01:15.341145+0000","last_peered":"2024-09-18T01:01:15.341145+0000","last_clean":"2024-09-18T01:01:15.341145+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T01:01:15.341145+0000","last_undegraded":"2024-09-18T01:01:15.341145+0000","last_fullsized":"2024-09-18T01:01:15.341145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:44.745709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036716330000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8918,"num_read_kb":44936,"num_write":410,"num_write_kb":13864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382260+0000","last_change":"2024-09-18T01:00:15.991240+0000","last_active":"2024-09-18T01:01:14.382260+0000","last_peered":"2024-09-18T01:01:14.382260+0000","last_clean":"2024-09-18T01:01:14.382260+0000","last_became_active":"2024-09-18T01:00:15.990930+0000","last_became_peered":"2024-09-18T01:00:15.990930+0000","last_unstale":"2024-09-18T01:01:14.382260+0000","last_undegraded":"2024-09-18T01:01:14.382260+0000","last_fullsized":"2024-09-18T01:01:14.382260+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:55.616429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"185'348","reported_seq":4491,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173966+0000","last_change":"2024-09-18T01:00:44.260854+0000","last_active":"2024-09-18T01:01:15.173966+0000","last_peered":"2024-09-18T01:01:15.173966+0000","last_clean":"2024-09-18T01:01:15.173966+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T01:01:15.173966+0000","last_undegraded":"2024-09-18T01:01:15.173966+0000","last_fullsized":"2024-09-18T01:01:15.173966+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:09.012851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019390989999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3644,"num_read_kb":27224,"num_write":289,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":399,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590664+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T01:01:15.590664+0000","last_peered":"2024-09-18T01:01:15.590664+0000","last_clean":"2024-09-18T01:01:15.590664+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T01:01:15.590664+0000","last_undegraded":"2024-09-18T01:01:15.590664+0000","last_fullsized":"2024-09-18T01:01:15.590664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"185'580","reported_seq":4494,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380809+0000","last_change":"2024-09-18T01:01:01.242483+0000","last_active":"2024-09-18T01:01:14.380809+0000","last_peered":"2024-09-18T01:01:14.380809+0000","last_clean":"2024-09-18T01:01:14.380809+0000","last_became_active":"2024-09-18T01:01:01.239957+0000","last_became_peered":"2024-09-18T01:01:01.239957+0000","last_unstale":"2024-09-18T01:01:14.380809+0000","last_undegraded":"2024-09-18T01:01:14.380809+0000","last_fullsized":"2024-09-18T01:01:14.380809+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:46.781823+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.024809728,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4830,"num_read_kb":33562,"num_write":800,"num_write_kb":24925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"186'946","reported_seq":6143,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941033+0000","last_change":"2024-09-18T01:01:03.958936+0000","last_active":"2024-09-18T01:01:14.941033+0000","last_peered":"2024-09-18T01:01:14.941033+0000","last_clean":"2024-09-18T01:01:14.941033+0000","last_became_active":"2024-09-18T01:01:03.955764+0000","last_became_peered":"2024-09-18T01:01:03.955764+0000","last_unstale":"2024-09-18T01:01:14.941033+0000","last_undegraded":"2024-09-18T01:01:14.941033+0000","last_fullsized":"2024-09-18T01:01:14.941033+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:25.024002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022401439999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7644,"num_read_kb":19178,"num_write":1076,"num_write_kb":33944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":99850853,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":56,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380143+0000","last_change":"2024-09-18T01:00:51.188692+0000","last_active":"2024-09-18T01:01:14.380143+0000","last_peered":"2024-09-18T01:01:14.380143+0000","last_clean":"2024-09-18T01:01:14.380143+0000","last_became_active":"2024-09-18T01:00:51.183713+0000","last_became_peered":"2024-09-18T01:00:51.183713+0000","last_unstale":"2024-09-18T01:01:14.380143+0000","last_undegraded":"2024-09-18T01:01:14.380143+0000","last_fullsized":"2024-09-18T01:01:14.380143+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:20:22.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"185'657","reported_seq":8382,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590875+0000","last_change":"2024-09-18T01:00:44.156905+0000","last_active":"2024-09-18T01:01:15.590875+0000","last_peered":"2024-09-18T01:01:15.590875+0000","last_clean":"2024-09-18T01:01:15.590875+0000","last_became_active":"2024-09-18T01:00:08.951992+0000","last_became_peered":"2024-09-18T01:00:08.951992+0000","last_unstale":"2024-09-18T01:01:15.590875+0000","last_undegraded":"2024-09-18T01:01:15.590875+0000","last_fullsized":"2024-09-18T01:01:15.590875+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:59.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030555019999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7663,"num_read_kb":29924,"num_write":748,"num_write_kb":19563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":15190015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381997+0000","last_change":"2024-09-18T01:00:04.976725+0000","last_active":"2024-09-18T01:01:14.381997+0000","last_peered":"2024-09-18T01:01:14.381997+0000","last_clean":"2024-09-18T01:01:14.381997+0000","last_became_active":"2024-09-18T01:00:04.976360+0000","last_became_peered":"2024-09-18T01:00:04.976360+0000","last_unstale":"2024-09-18T01:01:14.381997+0000","last_undegraded":"2024-09-18T01:01:14.381997+0000","last_fullsized":"2024-09-18T01:01:14.381997+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:11.159126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"184'548","reported_seq":9961,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173598+0000","last_change":"2024-09-18T01:01:01.957077+0000","last_active":"2024-09-18T01:01:15.173598+0000","last_peered":"2024-09-18T01:01:15.173598+0000","last_clean":"2024-09-18T01:01:15.173598+0000","last_became_active":"2024-09-18T01:01:01.953404+0000","last_became_peered":"2024-09-18T01:01:01.953404+0000","last_unstale":"2024-09-18T01:01:15.173598+0000","last_undegraded":"2024-09-18T01:01:15.173598+0000","last_fullsized":"2024-09-18T01:01:15.173598+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:03.050834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726180000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8919,"num_read_kb":48544,"num_write":559,"num_write_kb":18757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1056,"num_bytes_recovered":19034719,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379823+0000","last_change":"2024-09-18T01:00:15.994517+0000","last_active":"2024-09-18T01:01:14.379823+0000","last_peered":"2024-09-18T01:01:14.379823+0000","last_clean":"2024-09-18T01:01:14.379823+0000","last_became_active":"2024-09-18T01:00:15.994159+0000","last_became_peered":"2024-09-18T01:00:15.994159+0000","last_unstale":"2024-09-18T01:01:14.379823+0000","last_undegraded":"2024-09-18T01:01:14.379823+0000","last_fullsized":"2024-09-18T01:01:14.379823+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:06.775713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"186'722","reported_seq":5332,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340913+0000","last_change":"2024-09-18T01:01:01.952665+0000","last_active":"2024-09-18T01:01:15.340913+0000","last_peered":"2024-09-18T01:01:15.340913+0000","last_clean":"2024-09-18T01:01:15.340913+0000","last_became_active":"2024-09-18T01:01:01.948535+0000","last_became_peered":"2024-09-18T01:01:01.948535+0000","last_unstale":"2024-09-18T01:01:15.340913+0000","last_undegraded":"2024-09-18T01:01:15.340913+0000","last_fullsized":"2024-09-18T01:01:15.340913+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:39.959837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00139405,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5240,"num_read_kb":16439,"num_write":1146,"num_write_kb":52135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":12961092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":153,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380940+0000","last_change":"2024-09-18T01:00:58.232332+0000","last_active":"2024-09-18T01:01:14.380940+0000","last_peered":"2024-09-18T01:01:14.380940+0000","last_clean":"2024-09-18T01:01:14.380940+0000","last_became_active":"2024-09-18T01:00:58.229581+0000","last_became_peered":"2024-09-18T01:00:58.229581+0000","last_unstale":"2024-09-18T01:01:14.380940+0000","last_undegraded":"2024-09-18T01:01:14.380940+0000","last_fullsized":"2024-09-18T01:01:14.380940+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:05:38.818642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":551,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912328+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T01:01:14.912328+0000","last_peered":"2024-09-18T01:01:14.912328+0000","last_clean":"2024-09-18T01:01:14.912328+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T01:01:14.912328+0000","last_undegraded":"2024-09-18T01:01:14.912328+0000","last_fullsized":"2024-09-18T01:01:14.912328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"184'448","reported_seq":5879,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340789+0000","last_change":"2024-09-18T01:00:44.149901+0000","last_active":"2024-09-18T01:01:15.340789+0000","last_peered":"2024-09-18T01:01:15.340789+0000","last_clean":"2024-09-18T01:01:15.340789+0000","last_became_active":"2024-09-18T01:00:08.949384+0000","last_became_peered":"2024-09-18T01:00:08.949384+0000","last_unstale":"2024-09-18T01:01:15.340789+0000","last_undegraded":"2024-09-18T01:01:15.340789+0000","last_fullsized":"2024-09-18T01:01:15.340789+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:20.064612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033952990000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5722,"num_read_kb":51790,"num_write":458,"num_write_kb":39546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":32435091,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"185'408","reported_seq":3782,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379828+0000","last_change":"2024-09-18T01:00:44.145255+0000","last_active":"2024-09-18T01:01:14.379828+0000","last_peered":"2024-09-18T01:01:14.379828+0000","last_clean":"2024-09-18T01:01:14.379828+0000","last_became_active":"2024-09-18T01:00:18.997710+0000","last_became_peered":"2024-09-18T01:00:18.997710+0000","last_unstale":"2024-09-18T01:01:14.379828+0000","last_undegraded":"2024-09-18T01:01:14.379828+0000","last_fullsized":"2024-09-18T01:01:14.379828+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:36.454597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016382809999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3625,"num_read_kb":11553,"num_write":342,"num_write_kb":23969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":27148173,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"185'607","reported_seq":5628,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380626+0000","last_change":"2024-09-18T01:00:44.146011+0000","last_active":"2024-09-18T01:01:14.380626+0000","last_peered":"2024-09-18T01:01:14.380626+0000","last_clean":"2024-09-18T01:01:14.380626+0000","last_became_active":"2024-09-18T01:00:22.938344+0000","last_became_peered":"2024-09-18T01:00:22.938344+0000","last_unstale":"2024-09-18T01:01:14.380626+0000","last_undegraded":"2024-09-18T01:01:14.380626+0000","last_fullsized":"2024-09-18T01:01:14.380626+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:59:54.570595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021924589999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4754,"num_read_kb":46819,"num_write":596,"num_write_kb":17196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":6639875,"num_keys_recovered":85,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"185'515","reported_seq":5914,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382003+0000","last_change":"2024-09-18T01:00:44.262988+0000","last_active":"2024-09-18T01:01:14.382003+0000","last_peered":"2024-09-18T01:01:14.382003+0000","last_clean":"2024-09-18T01:01:14.382003+0000","last_became_active":"2024-09-18T01:00:10.954910+0000","last_became_peered":"2024-09-18T01:00:10.954910+0000","last_unstale":"2024-09-18T01:01:14.382003+0000","last_undegraded":"2024-09-18T01:01:14.382003+0000","last_fullsized":"2024-09-18T01:01:14.382003+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:45.069175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042875949999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5466,"num_read_kb":66445,"num_write":496,"num_write_kb":15031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15399944,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"185'566","reported_seq":7107,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941155+0000","last_change":"2024-09-18T01:01:02.956425+0000","last_active":"2024-09-18T01:01:14.941155+0000","last_peered":"2024-09-18T01:01:14.941155+0000","last_clean":"2024-09-18T01:01:14.941155+0000","last_became_active":"2024-09-18T01:01:02.953950+0000","last_became_peered":"2024-09-18T01:01:02.953950+0000","last_unstale":"2024-09-18T01:01:14.941155+0000","last_undegraded":"2024-09-18T01:01:14.941155+0000","last_fullsized":"2024-09-18T01:01:14.941155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:11.064860+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0026394880000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6401,"num_read_kb":47394,"num_write":633,"num_write_kb":22377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":560,"num_bytes_recovered":26783458,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"184'976","reported_seq":6804,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381748+0000","last_change":"2024-09-18T01:00:44.146438+0000","last_active":"2024-09-18T01:01:14.381748+0000","last_peered":"2024-09-18T01:01:14.381748+0000","last_clean":"2024-09-18T01:01:14.381748+0000","last_became_active":"2024-09-18T01:00:21.923279+0000","last_became_peered":"2024-09-18T01:00:21.923279+0000","last_unstale":"2024-09-18T01:01:14.381748+0000","last_undegraded":"2024-09-18T01:01:14.381748+0000","last_fullsized":"2024-09-18T01:01:14.381748+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:23.093467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026100659999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8636,"num_read_kb":40955,"num_write":1143,"num_write_kb":30743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":80305919,"num_keys_recovered":163,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"183'713","reported_seq":4745,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380694+0000","last_change":"2024-09-18T01:01:02.959668+0000","last_active":"2024-09-18T01:01:14.380694+0000","last_peered":"2024-09-18T01:01:14.380694+0000","last_clean":"2024-09-18T01:01:14.380694+0000","last_became_active":"2024-09-18T01:01:02.954337+0000","last_became_peered":"2024-09-18T01:01:02.954337+0000","last_unstale":"2024-09-18T01:01:14.380694+0000","last_undegraded":"2024-09-18T01:01:14.380694+0000","last_fullsized":"2024-09-18T01:01:14.380694+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:56:25.045514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0093305890000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4741,"num_read_kb":29100,"num_write":948,"num_write_kb":26466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":41587165,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"184'563","reported_seq":4380,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941359+0000","last_change":"2024-09-18T01:01:01.950143+0000","last_active":"2024-09-18T01:01:14.941359+0000","last_peered":"2024-09-18T01:01:14.941359+0000","last_clean":"2024-09-18T01:01:14.941359+0000","last_became_active":"2024-09-18T01:01:01.948267+0000","last_became_peered":"2024-09-18T01:01:01.948267+0000","last_unstale":"2024-09-18T01:01:14.941359+0000","last_undegraded":"2024-09-18T01:01:14.941359+0000","last_fullsized":"2024-09-18T01:01:14.941359+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:43:18.359056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024190230000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5311,"num_read_kb":28191,"num_write":598,"num_write_kb":9287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6320640,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"184'456","reported_seq":5302,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912243+0000","last_change":"2024-09-18T01:00:44.148986+0000","last_active":"2024-09-18T01:01:14.912243+0000","last_peered":"2024-09-18T01:01:14.912243+0000","last_clean":"2024-09-18T01:01:14.912243+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T01:01:14.912243+0000","last_undegraded":"2024-09-18T01:01:14.912243+0000","last_fullsized":"2024-09-18T01:01:14.912243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:48:20.609462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001443239,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":51584,"num_write":357,"num_write_kb":26802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"183'292","reported_seq":3444,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380096+0000","last_change":"2024-09-18T01:01:02.956466+0000","last_active":"2024-09-18T01:01:14.380096+0000","last_peered":"2024-09-18T01:01:14.380096+0000","last_clean":"2024-09-18T01:01:14.380096+0000","last_became_active":"2024-09-18T01:01:02.950361+0000","last_became_peered":"2024-09-18T01:01:02.950361+0000","last_unstale":"2024-09-18T01:01:14.380096+0000","last_undegraded":"2024-09-18T01:01:14.380096+0000","last_fullsized":"2024-09-18T01:01:14.380096+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:56:04.680353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033253200000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1897,"num_read_kb":13175,"num_write":246,"num_write_kb":14260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":15824068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"184'638","reported_seq":6909,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173643+0000","last_change":"2024-09-18T01:00:44.261997+0000","last_active":"2024-09-18T01:01:15.173643+0000","last_peered":"2024-09-18T01:01:15.173643+0000","last_clean":"2024-09-18T01:01:15.173643+0000","last_became_active":"2024-09-18T01:00:21.926814+0000","last_became_peered":"2024-09-18T01:00:21.926814+0000","last_unstale":"2024-09-18T01:01:15.173643+0000","last_undegraded":"2024-09-18T01:01:15.173643+0000","last_fullsized":"2024-09-18T01:01:15.173643+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:04.145432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002915932,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6954,"num_read_kb":17130,"num_write":754,"num_write_kb":11285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":32804395,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"185'521","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381219+0000","last_change":"2024-09-18T01:01:01.245238+0000","last_active":"2024-09-18T01:01:14.381219+0000","last_peered":"2024-09-18T01:01:14.381219+0000","last_clean":"2024-09-18T01:01:14.381219+0000","last_became_active":"2024-09-18T01:01:01.242541+0000","last_became_peered":"2024-09-18T01:01:01.242541+0000","last_unstale":"2024-09-18T01:01:14.381219+0000","last_undegraded":"2024-09-18T01:01:14.381219+0000","last_fullsized":"2024-09-18T01:01:14.381219+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:44.647014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027658140000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5114,"num_read_kb":26799,"num_write":585,"num_write_kb":11735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":9379374,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":482,"num_read":220594,"num_read_kb":1067291,"num_write":22261,"num_write_kb":688577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12013,"num_bytes_recovered":693846180,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19538,"ondisk_log_size":19538,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47592,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324248,"statfs":{"total":96636764160,"available":96588029952,"internally_reserved":0,"allocated":2572288,"data_stored":1237498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52344,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319496,"statfs":{"total":96636764160,"available":96583163904,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60064,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311776,"statfs":{"total":96636764160,"available":96575258624,"internally_reserved":0,"allocated":2568192,"data_stored":1237479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55920,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315920,"statfs":{"total":96636764160,"available":96579502080,"internally_reserved":0,"allocated":2584576,"data_stored":1243311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60080,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311760,"statfs":{"total":96636764160,"available":96575242240,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52344,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319496,"statfs":{"total":96636764160,"available":96583163904,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47600,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324240,"statfs":{"total":96636764160,"available":96588021760,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:01:19.089 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T01:01:19.089 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:01:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:01:19.399 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T01:01:19.418 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-18T01:01:17.946193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":482,"num_read":220710,"num_read_kb":1067391,"num_write":22422,"num_write_kb":690995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12021,"num_bytes_recovered":693846218,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19658,"ondisk_log_size":19658,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":427704,"kb_used_data":21312,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754547016,"statfs":{"total":773094113280,"available":772656144384,"internally_reserved":0,"allocated":21823488,"data_stored":11115522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":415944240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001933"},"pg_stats":[{"pgid":"3.1f","version":"186'621","reported_seq":4223,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590664+0000","last_change":"2024-09-18T01:00:44.155176+0000","last_active":"2024-09-18T01:01:15.590664+0000","last_peered":"2024-09-18T01:01:15.590664+0000","last_clean":"2024-09-18T01:01:15.590664+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T01:01:15.590664+0000","last_undegraded":"2024-09-18T01:01:15.590664+0000","last_fullsized":"2024-09-18T01:01:15.590664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:49.598474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001427074,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6437,"num_read_kb":23165,"num_write":614,"num_write_kb":11006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"186'646","reported_seq":5759,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590628+0000","last_change":"2024-09-18T01:01:01.248306+0000","last_active":"2024-09-18T01:01:15.590628+0000","last_peered":"2024-09-18T01:01:15.590628+0000","last_clean":"2024-09-18T01:01:15.590628+0000","last_became_active":"2024-09-18T01:01:01.245192+0000","last_became_peered":"2024-09-18T01:01:01.245192+0000","last_unstale":"2024-09-18T01:01:15.590628+0000","last_undegraded":"2024-09-18T01:01:15.590628+0000","last_fullsized":"2024-09-18T01:01:15.590628+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:50.236589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041620060000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4822,"num_read_kb":36392,"num_write":703,"num_write_kb":11559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":30825045,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"184'800","reported_seq":12971,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382040+0000","last_change":"2024-09-18T01:01:03.959251+0000","last_active":"2024-09-18T01:01:14.382040+0000","last_peered":"2024-09-18T01:01:14.382040+0000","last_clean":"2024-09-18T01:01:14.382040+0000","last_became_active":"2024-09-18T01:01:03.956061+0000","last_became_peered":"2024-09-18T01:01:03.956061+0000","last_unstale":"2024-09-18T01:01:14.382040+0000","last_undegraded":"2024-09-18T01:01:14.382040+0000","last_fullsized":"2024-09-18T01:01:14.382040+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:21:07.686876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034116569999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":13334,"num_read_kb":40113,"num_write":945,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":705,"num_bytes_recovered":22094834,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"186'663","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912412+0000","last_change":"2024-09-18T01:01:01.947287+0000","last_active":"2024-09-18T01:01:14.912412+0000","last_peered":"2024-09-18T01:01:14.912412+0000","last_clean":"2024-09-18T01:01:14.912412+0000","last_became_active":"2024-09-18T01:01:01.945747+0000","last_became_peered":"2024-09-18T01:01:01.945747+0000","last_unstale":"2024-09-18T01:01:14.912412+0000","last_undegraded":"2024-09-18T01:01:14.912412+0000","last_fullsized":"2024-09-18T01:01:14.912412+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:44:09.776110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002343262,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5693,"num_read_kb":28138,"num_write":736,"num_write_kb":11405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":5428683,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"186'1093","reported_seq":10876,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340934+0000","last_change":"2024-09-18T01:01:01.953195+0000","last_active":"2024-09-18T01:01:15.340934+0000","last_peered":"2024-09-18T01:01:15.340934+0000","last_clean":"2024-09-18T01:01:15.340934+0000","last_became_active":"2024-09-18T01:01:01.947957+0000","last_became_peered":"2024-09-18T01:01:01.947957+0000","last_unstale":"2024-09-18T01:01:15.340934+0000","last_undegraded":"2024-09-18T01:01:15.340934+0000","last_fullsized":"2024-09-18T01:01:15.340934+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:50:57.100165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003279527,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12686,"num_read_kb":40787,"num_write":1375,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":59340116,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"186'364","reported_seq":5144,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912518+0000","last_change":"2024-09-18T01:00:44.151068+0000","last_active":"2024-09-18T01:01:14.912518+0000","last_peered":"2024-09-18T01:01:14.912518+0000","last_clean":"2024-09-18T01:01:14.912518+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T01:01:14.912518+0000","last_undegraded":"2024-09-18T01:01:14.912518+0000","last_fullsized":"2024-09-18T01:01:14.912518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:24.750570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017298630000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5410,"num_read_kb":29581,"num_write":319,"num_write_kb":12768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"186'701","reported_seq":30888,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340790+0000","last_change":"2024-09-18T01:00:51.958851+0000","last_active":"2024-09-18T01:01:15.340790+0000","last_peered":"2024-09-18T01:01:15.340790+0000","last_clean":"2024-09-18T01:01:15.340790+0000","last_became_active":"2024-09-18T01:00:51.953781+0000","last_became_peered":"2024-09-18T01:00:51.953781+0000","last_unstale":"2024-09-18T01:01:15.340790+0000","last_undegraded":"2024-09-18T01:01:15.340790+0000","last_fullsized":"2024-09-18T01:01:15.340790+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:44.107525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001459178,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":30614,"num_read_kb":78260,"num_write":886,"num_write_kb":27079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":40291284,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"186'656","reported_seq":4017,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.591127+0000","last_change":"2024-09-18T01:00:44.157985+0000","last_active":"2024-09-18T01:01:15.591127+0000","last_peered":"2024-09-18T01:01:15.591127+0000","last_clean":"2024-09-18T01:01:15.591127+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T01:01:15.591127+0000","last_undegraded":"2024-09-18T01:01:15.591127+0000","last_fullsized":"2024-09-18T01:01:15.591127+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:27.075227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039694869999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4869,"num_read_kb":13814,"num_write":877,"num_write_kb":42194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"185'676","reported_seq":7355,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590783+0000","last_change":"2024-09-18T01:00:44.156705+0000","last_active":"2024-09-18T01:01:15.590783+0000","last_peered":"2024-09-18T01:01:15.590783+0000","last_clean":"2024-09-18T01:01:15.590783+0000","last_became_active":"2024-09-18T01:00:20.017711+0000","last_became_peered":"2024-09-18T01:00:20.017711+0000","last_unstale":"2024-09-18T01:01:15.590783+0000","last_undegraded":"2024-09-18T01:01:15.590783+0000","last_fullsized":"2024-09-18T01:01:15.590783+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:49.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028584439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7284,"num_read_kb":46982,"num_write":773,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":13346337,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"186'641","reported_seq":8980,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380983+0000","last_change":"2024-09-18T01:01:01.955368+0000","last_active":"2024-09-18T01:01:14.380983+0000","last_peered":"2024-09-18T01:01:14.380983+0000","last_clean":"2024-09-18T01:01:14.380983+0000","last_became_active":"2024-09-18T01:01:01.952512+0000","last_became_peered":"2024-09-18T01:01:01.952512+0000","last_unstale":"2024-09-18T01:01:14.380983+0000","last_undegraded":"2024-09-18T01:01:14.380983+0000","last_fullsized":"2024-09-18T01:01:14.380983+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:36:14.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887849999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8507,"num_read_kb":37252,"num_write":699,"num_write_kb":22150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":946,"num_bytes_recovered":37601010,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"184'519","reported_seq":3007,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941255+0000","last_change":"2024-09-18T01:01:02.959624+0000","last_active":"2024-09-18T01:01:14.941255+0000","last_peered":"2024-09-18T01:01:14.941255+0000","last_clean":"2024-09-18T01:01:14.941255+0000","last_became_active":"2024-09-18T01:01:02.956655+0000","last_became_peered":"2024-09-18T01:01:02.956655+0000","last_unstale":"2024-09-18T01:01:14.941255+0000","last_undegraded":"2024-09-18T01:01:14.941255+0000","last_fullsized":"2024-09-18T01:01:14.941255+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:53:56.284343+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010838409,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2322,"num_read_kb":15300,"num_write":809,"num_write_kb":48070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":22291504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"181'371","reported_seq":3210,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381490+0000","last_change":"2024-09-18T01:01:01.948935+0000","last_active":"2024-09-18T01:01:14.381490+0000","last_peered":"2024-09-18T01:01:14.381490+0000","last_clean":"2024-09-18T01:01:14.381490+0000","last_became_active":"2024-09-18T01:01:01.947158+0000","last_became_peered":"2024-09-18T01:01:01.947158+0000","last_unstale":"2024-09-18T01:01:14.381490+0000","last_undegraded":"2024-09-18T01:01:14.381490+0000","last_fullsized":"2024-09-18T01:01:14.381490+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:05.169088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013361650000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2452,"num_read_kb":3816,"num_write":339,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":3474495,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":127,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382692+0000","last_change":"2024-09-18T01:00:59.238970+0000","last_active":"2024-09-18T01:01:14.382692+0000","last_peered":"2024-09-18T01:01:14.382692+0000","last_clean":"2024-09-18T01:01:14.382692+0000","last_became_active":"2024-09-18T01:00:59.238106+0000","last_became_peered":"2024-09-18T01:00:59.238106+0000","last_unstale":"2024-09-18T01:01:14.382692+0000","last_undegraded":"2024-09-18T01:01:14.382692+0000","last_fullsized":"2024-09-18T01:01:14.382692+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:15:05.506655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"186'777","reported_seq":4883,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.341002+0000","last_change":"2024-09-18T01:00:44.148713+0000","last_active":"2024-09-18T01:01:15.341002+0000","last_peered":"2024-09-18T01:01:15.341002+0000","last_clean":"2024-09-18T01:01:15.341002+0000","last_became_active":"2024-09-18T01:00:11.936218+0000","last_became_peered":"2024-09-18T01:00:11.936218+0000","last_unstale":"2024-09-18T01:01:15.341002+0000","last_undegraded":"2024-09-18T01:01:15.341002+0000","last_fullsized":"2024-09-18T01:01:15.341002+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:24.939297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002351681,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5488,"num_read_kb":22949,"num_write":1002,"num_write_kb":4650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":2805113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":412,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340720+0000","last_change":"2024-09-18T01:00:59.233973+0000","last_active":"2024-09-18T01:01:15.340720+0000","last_peered":"2024-09-18T01:01:15.340720+0000","last_clean":"2024-09-18T01:01:15.340720+0000","last_became_active":"2024-09-18T01:00:59.231756+0000","last_became_peered":"2024-09-18T01:00:59.231756+0000","last_unstale":"2024-09-18T01:01:15.340720+0000","last_undegraded":"2024-09-18T01:01:15.340720+0000","last_fullsized":"2024-09-18T01:01:15.340720+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:50:54.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"185'506","reported_seq":8883,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.341145+0000","last_change":"2024-09-18T01:00:44.150561+0000","last_active":"2024-09-18T01:01:15.341145+0000","last_peered":"2024-09-18T01:01:15.341145+0000","last_clean":"2024-09-18T01:01:15.341145+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T01:01:15.341145+0000","last_undegraded":"2024-09-18T01:01:15.341145+0000","last_fullsized":"2024-09-18T01:01:15.341145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:44.745709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036716330000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8918,"num_read_kb":44936,"num_write":410,"num_write_kb":13864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382260+0000","last_change":"2024-09-18T01:00:15.991240+0000","last_active":"2024-09-18T01:01:14.382260+0000","last_peered":"2024-09-18T01:01:14.382260+0000","last_clean":"2024-09-18T01:01:14.382260+0000","last_became_active":"2024-09-18T01:00:15.990930+0000","last_became_peered":"2024-09-18T01:00:15.990930+0000","last_unstale":"2024-09-18T01:01:14.382260+0000","last_undegraded":"2024-09-18T01:01:14.382260+0000","last_fullsized":"2024-09-18T01:01:14.382260+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:55.616429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"185'348","reported_seq":4491,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173966+0000","last_change":"2024-09-18T01:00:44.260854+0000","last_active":"2024-09-18T01:01:15.173966+0000","last_peered":"2024-09-18T01:01:15.173966+0000","last_clean":"2024-09-18T01:01:15.173966+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T01:01:15.173966+0000","last_undegraded":"2024-09-18T01:01:15.173966+0000","last_fullsized":"2024-09-18T01:01:15.173966+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:09.012851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019390989999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3644,"num_read_kb":27224,"num_write":289,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":399,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590664+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T01:01:15.590664+0000","last_peered":"2024-09-18T01:01:15.590664+0000","last_clean":"2024-09-18T01:01:15.590664+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T01:01:15.590664+0000","last_undegraded":"2024-09-18T01:01:15.590664+0000","last_fullsized":"2024-09-18T01:01:15.590664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"185'580","reported_seq":4494,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380809+0000","last_change":"2024-09-18T01:01:01.242483+0000","last_active":"2024-09-18T01:01:14.380809+0000","last_peered":"2024-09-18T01:01:14.380809+0000","last_clean":"2024-09-18T01:01:14.380809+0000","last_became_active":"2024-09-18T01:01:01.239957+0000","last_became_peered":"2024-09-18T01:01:01.239957+0000","last_unstale":"2024-09-18T01:01:14.380809+0000","last_undegraded":"2024-09-18T01:01:14.380809+0000","last_fullsized":"2024-09-18T01:01:14.380809+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:46.781823+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.024809728,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4830,"num_read_kb":33562,"num_write":800,"num_write_kb":24925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"186'946","reported_seq":6143,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941033+0000","last_change":"2024-09-18T01:01:03.958936+0000","last_active":"2024-09-18T01:01:14.941033+0000","last_peered":"2024-09-18T01:01:14.941033+0000","last_clean":"2024-09-18T01:01:14.941033+0000","last_became_active":"2024-09-18T01:01:03.955764+0000","last_became_peered":"2024-09-18T01:01:03.955764+0000","last_unstale":"2024-09-18T01:01:14.941033+0000","last_undegraded":"2024-09-18T01:01:14.941033+0000","last_fullsized":"2024-09-18T01:01:14.941033+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:25.024002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022401439999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7644,"num_read_kb":19178,"num_write":1076,"num_write_kb":33944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":99850853,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":56,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380143+0000","last_change":"2024-09-18T01:00:51.188692+0000","last_active":"2024-09-18T01:01:14.380143+0000","last_peered":"2024-09-18T01:01:14.380143+0000","last_clean":"2024-09-18T01:01:14.380143+0000","last_became_active":"2024-09-18T01:00:51.183713+0000","last_became_peered":"2024-09-18T01:00:51.183713+0000","last_unstale":"2024-09-18T01:01:14.380143+0000","last_undegraded":"2024-09-18T01:01:14.380143+0000","last_fullsized":"2024-09-18T01:01:14.380143+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:20:22.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"185'657","reported_seq":8382,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590875+0000","last_change":"2024-09-18T01:00:44.156905+0000","last_active":"2024-09-18T01:01:15.590875+0000","last_peered":"2024-09-18T01:01:15.590875+0000","last_clean":"2024-09-18T01:01:15.590875+0000","last_became_active":"2024-09-18T01:00:08.951992+0000","last_became_peered":"2024-09-18T01:00:08.951992+0000","last_unstale":"2024-09-18T01:01:15.590875+0000","last_undegraded":"2024-09-18T01:01:15.590875+0000","last_fullsized":"2024-09-18T01:01:15.590875+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:59.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030555019999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7663,"num_read_kb":29924,"num_write":748,"num_write_kb":19563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":15190015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381997+0000","last_change":"2024-09-18T01:00:04.976725+0000","last_active":"2024-09-18T01:01:14.381997+0000","last_peered":"2024-09-18T01:01:14.381997+0000","last_clean":"2024-09-18T01:01:14.381997+0000","last_became_active":"2024-09-18T01:00:04.976360+0000","last_became_peered":"2024-09-18T01:00:04.976360+0000","last_unstale":"2024-09-18T01:01:14.381997+0000","last_undegraded":"2024-09-18T01:01:14.381997+0000","last_fullsized":"2024-09-18T01:01:14.381997+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:11.159126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"184'548","reported_seq":9961,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173598+0000","last_change":"2024-09-18T01:01:01.957077+0000","last_active":"2024-09-18T01:01:15.173598+0000","last_peered":"2024-09-18T01:01:15.173598+0000","last_clean":"2024-09-18T01:01:15.173598+0000","last_became_active":"2024-09-18T01:01:01.953404+0000","last_became_peered":"2024-09-18T01:01:01.953404+0000","last_unstale":"2024-09-18T01:01:15.173598+0000","last_undegraded":"2024-09-18T01:01:15.173598+0000","last_fullsized":"2024-09-18T01:01:15.173598+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:03.050834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726180000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8919,"num_read_kb":48544,"num_write":559,"num_write_kb":18757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1056,"num_bytes_recovered":19034719,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379823+0000","last_change":"2024-09-18T01:00:15.994517+0000","last_active":"2024-09-18T01:01:14.379823+0000","last_peered":"2024-09-18T01:01:14.379823+0000","last_clean":"2024-09-18T01:01:14.379823+0000","last_became_active":"2024-09-18T01:00:15.994159+0000","last_became_peered":"2024-09-18T01:00:15.994159+0000","last_unstale":"2024-09-18T01:01:14.379823+0000","last_undegraded":"2024-09-18T01:01:14.379823+0000","last_fullsized":"2024-09-18T01:01:14.379823+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:06.775713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"186'722","reported_seq":5332,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340913+0000","last_change":"2024-09-18T01:01:01.952665+0000","last_active":"2024-09-18T01:01:15.340913+0000","last_peered":"2024-09-18T01:01:15.340913+0000","last_clean":"2024-09-18T01:01:15.340913+0000","last_became_active":"2024-09-18T01:01:01.948535+0000","last_became_peered":"2024-09-18T01:01:01.948535+0000","last_unstale":"2024-09-18T01:01:15.340913+0000","last_undegraded":"2024-09-18T01:01:15.340913+0000","last_fullsized":"2024-09-18T01:01:15.340913+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:39.959837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00139405,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5240,"num_read_kb":16439,"num_write":1146,"num_write_kb":52135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":12961092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":153,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380940+0000","last_change":"2024-09-18T01:00:58.232332+0000","last_active":"2024-09-18T01:01:14.380940+0000","last_peered":"2024-09-18T01:01:14.380940+0000","last_clean":"2024-09-18T01:01:14.380940+0000","last_became_active":"2024-09-18T01:00:58.229581+0000","last_became_peered":"2024-09-18T01:00:58.229581+0000","last_unstale":"2024-09-18T01:01:14.380940+0000","last_undegraded":"2024-09-18T01:01:14.380940+0000","last_fullsized":"2024-09-18T01:01:14.380940+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:05:38.818642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":551,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912328+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T01:01:14.912328+0000","last_peered":"2024-09-18T01:01:14.912328+0000","last_clean":"2024-09-18T01:01:14.912328+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T01:01:14.912328+0000","last_undegraded":"2024-09-18T01:01:14.912328+0000","last_fullsized":"2024-09-18T01:01:14.912328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"184'448","reported_seq":5879,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340789+0000","last_change":"2024-09-18T01:00:44.149901+0000","last_active":"2024-09-18T01:01:15.340789+0000","last_peered":"2024-09-18T01:01:15.340789+0000","last_clean":"2024-09-18T01:01:15.340789+0000","last_became_active":"2024-09-18T01:00:08.949384+0000","last_became_peered":"2024-09-18T01:00:08.949384+0000","last_unstale":"2024-09-18T01:01:15.340789+0000","last_undegraded":"2024-09-18T01:01:15.340789+0000","last_fullsized":"2024-09-18T01:01:15.340789+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:20.064612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033952990000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5722,"num_read_kb":51790,"num_write":458,"num_write_kb":39546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":32435091,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"185'408","reported_seq":3782,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379828+0000","last_change":"2024-09-18T01:00:44.145255+0000","last_active":"2024-09-18T01:01:14.379828+0000","last_peered":"2024-09-18T01:01:14.379828+0000","last_clean":"2024-09-18T01:01:14.379828+0000","last_became_active":"2024-09-18T01:00:18.997710+0000","last_became_peered":"2024-09-18T01:00:18.997710+0000","last_unstale":"2024-09-18T01:01:14.379828+0000","last_undegraded":"2024-09-18T01:01:14.379828+0000","last_fullsized":"2024-09-18T01:01:14.379828+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:36.454597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016382809999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3625,"num_read_kb":11553,"num_write":342,"num_write_kb":23969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":27148173,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"185'607","reported_seq":5628,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380626+0000","last_change":"2024-09-18T01:00:44.146011+0000","last_active":"2024-09-18T01:01:14.380626+0000","last_peered":"2024-09-18T01:01:14.380626+0000","last_clean":"2024-09-18T01:01:14.380626+0000","last_became_active":"2024-09-18T01:00:22.938344+0000","last_became_peered":"2024-09-18T01:00:22.938344+0000","last_unstale":"2024-09-18T01:01:14.380626+0000","last_undegraded":"2024-09-18T01:01:14.380626+0000","last_fullsized":"2024-09-18T01:01:14.380626+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:59:54.570595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021924589999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4754,"num_read_kb":46819,"num_write":596,"num_write_kb":17196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":6639875,"num_keys_recovered":85,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"185'515","reported_seq":5914,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382003+0000","last_change":"2024-09-18T01:00:44.262988+0000","last_active":"2024-09-18T01:01:14.382003+0000","last_peered":"2024-09-18T01:01:14.382003+0000","last_clean":"2024-09-18T01:01:14.382003+0000","last_became_active":"2024-09-18T01:00:10.954910+0000","last_became_peered":"2024-09-18T01:00:10.954910+0000","last_unstale":"2024-09-18T01:01:14.382003+0000","last_undegraded":"2024-09-18T01:01:14.382003+0000","last_fullsized":"2024-09-18T01:01:14.382003+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:45.069175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042875949999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5466,"num_read_kb":66445,"num_write":496,"num_write_kb":15031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15399944,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"185'566","reported_seq":7107,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941155+0000","last_change":"2024-09-18T01:01:02.956425+0000","last_active":"2024-09-18T01:01:14.941155+0000","last_peered":"2024-09-18T01:01:14.941155+0000","last_clean":"2024-09-18T01:01:14.941155+0000","last_became_active":"2024-09-18T01:01:02.953950+0000","last_became_peered":"2024-09-18T01:01:02.953950+0000","last_unstale":"2024-09-18T01:01:14.941155+0000","last_undegraded":"2024-09-18T01:01:14.941155+0000","last_fullsized":"2024-09-18T01:01:14.941155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:11.064860+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0026394880000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6401,"num_read_kb":47394,"num_write":633,"num_write_kb":22377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":560,"num_bytes_recovered":26783458,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"184'976","reported_seq":6804,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381748+0000","last_change":"2024-09-18T01:00:44.146438+0000","last_active":"2024-09-18T01:01:14.381748+0000","last_peered":"2024-09-18T01:01:14.381748+0000","last_clean":"2024-09-18T01:01:14.381748+0000","last_became_active":"2024-09-18T01:00:21.923279+0000","last_became_peered":"2024-09-18T01:00:21.923279+0000","last_unstale":"2024-09-18T01:01:14.381748+0000","last_undegraded":"2024-09-18T01:01:14.381748+0000","last_fullsized":"2024-09-18T01:01:14.381748+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:23.093467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026100659999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8636,"num_read_kb":40955,"num_write":1143,"num_write_kb":30743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":80305919,"num_keys_recovered":163,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"183'713","reported_seq":4745,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380694+0000","last_change":"2024-09-18T01:01:02.959668+0000","last_active":"2024-09-18T01:01:14.380694+0000","last_peered":"2024-09-18T01:01:14.380694+0000","last_clean":"2024-09-18T01:01:14.380694+0000","last_became_active":"2024-09-18T01:01:02.954337+0000","last_became_peered":"2024-09-18T01:01:02.954337+0000","last_unstale":"2024-09-18T01:01:14.380694+0000","last_undegraded":"2024-09-18T01:01:14.380694+0000","last_fullsized":"2024-09-18T01:01:14.380694+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:56:25.045514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0093305890000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4741,"num_read_kb":29100,"num_write":948,"num_write_kb":26466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":41587165,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"184'563","reported_seq":4380,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941359+0000","last_change":"2024-09-18T01:01:01.950143+0000","last_active":"2024-09-18T01:01:14.941359+0000","last_peered":"2024-09-18T01:01:14.941359+0000","last_clean":"2024-09-18T01:01:14.941359+0000","last_became_active":"2024-09-18T01:01:01.948267+0000","last_became_peered":"2024-09-18T01:01:01.948267+0000","last_unstale":"2024-09-18T01:01:14.941359+0000","last_undegraded":"2024-09-18T01:01:14.941359+0000","last_fullsized":"2024-09-18T01:01:14.941359+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:43:18.359056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024190230000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5311,"num_read_kb":28191,"num_write":598,"num_write_kb":9287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6320640,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"184'456","reported_seq":5302,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912243+0000","last_change":"2024-09-18T01:00:44.148986+0000","last_active":"2024-09-18T01:01:14.912243+0000","last_peered":"2024-09-18T01:01:14.912243+0000","last_clean":"2024-09-18T01:01:14.912243+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T01:01:14.912243+0000","last_undegraded":"2024-09-18T01:01:14.912243+0000","last_fullsized":"2024-09-18T01:01:14.912243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:48:20.609462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001443239,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":51584,"num_write":357,"num_write_kb":26802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"183'292","reported_seq":3444,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380096+0000","last_change":"2024-09-18T01:01:02.956466+0000","last_active":"2024-09-18T01:01:14.380096+0000","last_peered":"2024-09-18T01:01:14.380096+0000","last_clean":"2024-09-18T01:01:14.380096+0000","last_became_active":"2024-09-18T01:01:02.950361+0000","last_became_peered":"2024-09-18T01:01:02.950361+0000","last_unstale":"2024-09-18T01:01:14.380096+0000","last_undegraded":"2024-09-18T01:01:14.380096+0000","last_fullsized":"2024-09-18T01:01:14.380096+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:56:04.680353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033253200000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1897,"num_read_kb":13175,"num_write":246,"num_write_kb":14260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":15824068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"184'638","reported_seq":6909,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173643+0000","last_change":"2024-09-18T01:00:44.261997+0000","last_active":"2024-09-18T01:01:15.173643+0000","last_peered":"2024-09-18T01:01:15.173643+0000","last_clean":"2024-09-18T01:01:15.173643+0000","last_became_active":"2024-09-18T01:00:21.926814+0000","last_became_peered":"2024-09-18T01:00:21.926814+0000","last_unstale":"2024-09-18T01:01:15.173643+0000","last_undegraded":"2024-09-18T01:01:15.173643+0000","last_fullsized":"2024-09-18T01:01:15.173643+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:04.145432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002915932,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6954,"num_read_kb":17130,"num_write":754,"num_write_kb":11285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":32804395,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"185'521","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381219+0000","last_change":"2024-09-18T01:01:01.245238+0000","last_active":"2024-09-18T01:01:14.381219+0000","last_peered":"2024-09-18T01:01:14.381219+0000","last_clean":"2024-09-18T01:01:14.381219+0000","last_became_active":"2024-09-18T01:01:01.242541+0000","last_became_peered":"2024-09-18T01:01:01.242541+0000","last_unstale":"2024-09-18T01:01:14.381219+0000","last_undegraded":"2024-09-18T01:01:14.381219+0000","last_fullsized":"2024-09-18T01:01:14.381219+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:44.647014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027658140000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5114,"num_read_kb":26799,"num_write":585,"num_write_kb":11735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":9379374,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":482,"num_read":220594,"num_read_kb":1067291,"num_write":22261,"num_write_kb":688577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12013,"num_bytes_recovered":693846180,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19538,"ondisk_log_size":19538,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47592,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324248,"statfs":{"total":96636764160,"available":96588029952,"internally_reserved":0,"allocated":2572288,"data_stored":1237498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52344,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319496,"statfs":{"total":96636764160,"available":96583163904,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60064,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311776,"statfs":{"total":96636764160,"available":96575258624,"internally_reserved":0,"allocated":2568192,"data_stored":1237479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55920,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315920,"statfs":{"total":96636764160,"available":96579502080,"internally_reserved":0,"allocated":2584576,"data_stored":1243311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60080,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311760,"statfs":{"total":96636764160,"available":96575242240,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52344,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319496,"statfs":{"total":96636764160,"available":96583163904,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47600,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324240,"statfs":{"total":96636764160,"available":96588021760,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:01:19.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:01:19.715 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:01:19.716 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":214,"fsid":"82a96d76-ba83-49e2-a985-84604bd599ba","created":"2024-09-18T00:57:07.138732+0000","modified":"2024-09-18T01:01:14.367493+0000","last_up_change":"2024-09-18T00:57:20.258605+0000","last_in_change":"2024-09-18T01:00:58.211018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:57:21.962521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T00:57:25.721733+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T00:57:35.599413+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":186,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f359ce5-6f6e-4161-8554-8ac028347274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6801","nonce":199763443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6803","nonce":199763443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6807","nonce":199763443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":199763443},{"type":"v1","addr":"172.21.15.42:6805","nonce":199763443}]},"public_addr":"172.21.15.42:6801/199763443","cluster_addr":"172.21.15.42:6803/199763443","heartbeat_back_addr":"172.21.15.42:6807/199763443","heartbeat_front_addr":"172.21.15.42:6805/199763443","state":["exists","up"]},{"osd":1,"uuid":"48687611-1556-4b6d-86af-7872494a91d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6809","nonce":345149800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6811","nonce":345149800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6815","nonce":345149800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":345149800},{"type":"v1","addr":"172.21.15.42:6813","nonce":345149800}]},"public_addr":"172.21.15.42:6809/345149800","cluster_addr":"172.21.15.42:6811/345149800","heartbeat_back_addr":"172.21.15.42:6815/345149800","heartbeat_front_addr":"172.21.15.42:6813/345149800","state":["exists","up"]},{"osd":2,"uuid":"9e6b1f90-ce9b-46d4-92fe-d457f7e415d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6825","nonce":431239931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6827","nonce":431239931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6831","nonce":431239931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":431239931},{"type":"v1","addr":"172.21.15.42:6829","nonce":431239931}]},"public_addr":"172.21.15.42:6825/431239931","cluster_addr":"172.21.15.42:6827/431239931","heartbeat_back_addr":"172.21.15.42:6831/431239931","heartbeat_front_addr":"172.21.15.42:6829/431239931","state":["exists","up"]},{"osd":3,"uuid":"dfbb53ce-a8b6-4f8c-889b-12ad51cfaca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6817","nonce":1775474180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6819","nonce":1775474180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6823","nonce":1775474180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":1775474180},{"type":"v1","addr":"172.21.15.42:6821","nonce":1775474180}]},"public_addr":"172.21.15.42:6817/1775474180","cluster_addr":"172.21.15.42:6819/1775474180","heartbeat_back_addr":"172.21.15.42:6823/1775474180","heartbeat_front_addr":"172.21.15.42:6821/1775474180","state":["exists","up"]},{"osd":4,"uuid":"802ccbeb-7351-4b38-832a-695117c7e493","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6809","nonce":4198401207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6811","nonce":4198401207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6815","nonce":4198401207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":4198401207},{"type":"v1","addr":"172.21.15.195:6813","nonce":4198401207}]},"public_addr":"172.21.15.195:6809/4198401207","cluster_addr":"172.21.15.195:6811/4198401207","heartbeat_back_addr":"172.21.15.195:6815/4198401207","heartbeat_front_addr":"172.21.15.195:6813/4198401207","state":["exists","up"]},{"osd":5,"uuid":"d958b3da-efa1-4d86-b81d-e78874441437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6825","nonce":2599825823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6827","nonce":2599825823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6831","nonce":2599825823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":2599825823},{"type":"v1","addr":"172.21.15.195:6829","nonce":2599825823}]},"public_addr":"172.21.15.195:6825/2599825823","cluster_addr":"172.21.15.195:6827/2599825823","heartbeat_back_addr":"172.21.15.195:6831/2599825823","heartbeat_front_addr":"172.21.15.195:6829/2599825823","state":["exists","up"]},{"osd":6,"uuid":"51b3cea8-c686-4fd3-b3f4-ba936cf26b64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6801","nonce":3550656451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6803","nonce":3550656451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6807","nonce":3550656451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":3550656451},{"type":"v1","addr":"172.21.15.195:6805","nonce":3550656451}]},"public_addr":"172.21.15.195:6801/3550656451","cluster_addr":"172.21.15.195:6803/3550656451","heartbeat_back_addr":"172.21.15.195:6807/3550656451","heartbeat_front_addr":"172.21.15.195:6805/3550656451","state":["exists","up"]},{"osd":7,"uuid":"b02ead1c-837a-4b07-9ad1-4e903ed5bc4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6817","nonce":2993352359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6819","nonce":2993352359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6823","nonce":2993352359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2993352359},{"type":"v1","addr":"172.21.15.195:6821","nonce":2993352359}]},"public_addr":"172.21.15.195:6817/2993352359","cluster_addr":"172.21.15.195:6819/2993352359","heartbeat_back_addr":"172.21.15.195:6823/2993352359","heartbeat_front_addr":"172.21.15.195:6821/2993352359","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.896477+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.941756+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.560180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.077394+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.996264+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.511686+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:17.411051+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:57:16.888092+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:01:20.734 INFO:tasks.ceph:Scrubbing osd.0 2024-09-18T01:01:20.734 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:01:20.900 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:01:20.916 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-18T01:01:21.212 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 0 to deep-scrub 2024-09-18T01:01:21.231 INFO:tasks.ceph:Scrubbing osd.1 2024-09-18T01:01:21.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:01:21.375 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:01:21.376 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:01:21.391 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-18T01:01:21.682 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 1 to deep-scrub 2024-09-18T01:01:21.702 INFO:tasks.ceph:Scrubbing osd.2 2024-09-18T01:01:21.702 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:01:21.847 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:01:21.848 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:01:21.863 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-18T01:01:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 2 to deep-scrub 2024-09-18T01:01:22.298 INFO:tasks.ceph:Scrubbing osd.3 2024-09-18T01:01:22.298 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:01:22.461 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:01:22.461 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:01:22.461 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:01:22.479 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-18T01:01:22.775 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 3 to deep-scrub 2024-09-18T01:01:22.795 INFO:tasks.ceph:Scrubbing osd.4 2024-09-18T01:01:22.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:01:22.940 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:01:22.940 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:01:22.940 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:01:22.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-18T01:01:23.236 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 4 to deep-scrub 2024-09-18T01:01:23.252 INFO:tasks.ceph:Scrubbing osd.5 2024-09-18T01:01:23.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:01:23.401 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:01:23.402 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:01:23.402 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:01:23.416 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-18T01:01:23.696 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 5 to deep-scrub 2024-09-18T01:01:23.714 INFO:tasks.ceph:Scrubbing osd.6 2024-09-18T01:01:23.715 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:01:23.858 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:01:23.858 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:01:23.858 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:01:23.873 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-18T01:01:24.163 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 6 to deep-scrub 2024-09-18T01:01:24.183 INFO:tasks.ceph:Scrubbing osd.7 2024-09-18T01:01:24.183 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:01:24.335 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-18T01:01:24.335 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:01:24.335 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-18T01:01:24.355 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-18T01:01:24.639 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 7 to deep-scrub 2024-09-18T01:01:24.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:01:24.958 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:01:24.959 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T01:01:24.976 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-18T01:01:23.948149+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":482,"num_read":220710,"num_read_kb":1067391,"num_write":22422,"num_write_kb":690995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12021,"num_bytes_recovered":693846218,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19658,"ondisk_log_size":19658,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":427720,"kb_used_data":21336,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754547000,"statfs":{"total":773094113280,"available":772656128000,"internally_reserved":0,"allocated":21848064,"data_stored":11127148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":415944240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003360"},"pg_stats":[{"pgid":"3.1f","version":"186'621","reported_seq":4223,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590664+0000","last_change":"2024-09-18T01:00:44.155176+0000","last_active":"2024-09-18T01:01:15.590664+0000","last_peered":"2024-09-18T01:01:15.590664+0000","last_clean":"2024-09-18T01:01:15.590664+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T01:01:15.590664+0000","last_undegraded":"2024-09-18T01:01:15.590664+0000","last_fullsized":"2024-09-18T01:01:15.590664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:49.598474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001427074,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6437,"num_read_kb":23165,"num_write":614,"num_write_kb":11006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"186'646","reported_seq":5759,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590628+0000","last_change":"2024-09-18T01:01:01.248306+0000","last_active":"2024-09-18T01:01:15.590628+0000","last_peered":"2024-09-18T01:01:15.590628+0000","last_clean":"2024-09-18T01:01:15.590628+0000","last_became_active":"2024-09-18T01:01:01.245192+0000","last_became_peered":"2024-09-18T01:01:01.245192+0000","last_unstale":"2024-09-18T01:01:15.590628+0000","last_undegraded":"2024-09-18T01:01:15.590628+0000","last_fullsized":"2024-09-18T01:01:15.590628+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:50.236589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041620060000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4822,"num_read_kb":36392,"num_write":703,"num_write_kb":11559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":30825045,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"184'800","reported_seq":12971,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382040+0000","last_change":"2024-09-18T01:01:03.959251+0000","last_active":"2024-09-18T01:01:14.382040+0000","last_peered":"2024-09-18T01:01:14.382040+0000","last_clean":"2024-09-18T01:01:14.382040+0000","last_became_active":"2024-09-18T01:01:03.956061+0000","last_became_peered":"2024-09-18T01:01:03.956061+0000","last_unstale":"2024-09-18T01:01:14.382040+0000","last_undegraded":"2024-09-18T01:01:14.382040+0000","last_fullsized":"2024-09-18T01:01:14.382040+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:21:07.686876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034116569999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":13334,"num_read_kb":40113,"num_write":945,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":705,"num_bytes_recovered":22094834,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"186'663","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912412+0000","last_change":"2024-09-18T01:01:01.947287+0000","last_active":"2024-09-18T01:01:14.912412+0000","last_peered":"2024-09-18T01:01:14.912412+0000","last_clean":"2024-09-18T01:01:14.912412+0000","last_became_active":"2024-09-18T01:01:01.945747+0000","last_became_peered":"2024-09-18T01:01:01.945747+0000","last_unstale":"2024-09-18T01:01:14.912412+0000","last_undegraded":"2024-09-18T01:01:14.912412+0000","last_fullsized":"2024-09-18T01:01:14.912412+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:44:09.776110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002343262,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5693,"num_read_kb":28138,"num_write":736,"num_write_kb":11405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":5428683,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"186'1093","reported_seq":10876,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340934+0000","last_change":"2024-09-18T01:01:01.953195+0000","last_active":"2024-09-18T01:01:15.340934+0000","last_peered":"2024-09-18T01:01:15.340934+0000","last_clean":"2024-09-18T01:01:15.340934+0000","last_became_active":"2024-09-18T01:01:01.947957+0000","last_became_peered":"2024-09-18T01:01:01.947957+0000","last_unstale":"2024-09-18T01:01:15.340934+0000","last_undegraded":"2024-09-18T01:01:15.340934+0000","last_fullsized":"2024-09-18T01:01:15.340934+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:50:57.100165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003279527,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12686,"num_read_kb":40787,"num_write":1375,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":59340116,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"186'364","reported_seq":5144,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912518+0000","last_change":"2024-09-18T01:00:44.151068+0000","last_active":"2024-09-18T01:01:14.912518+0000","last_peered":"2024-09-18T01:01:14.912518+0000","last_clean":"2024-09-18T01:01:14.912518+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T01:01:14.912518+0000","last_undegraded":"2024-09-18T01:01:14.912518+0000","last_fullsized":"2024-09-18T01:01:14.912518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:24.750570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017298630000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5410,"num_read_kb":29581,"num_write":319,"num_write_kb":12768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"186'701","reported_seq":30888,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340790+0000","last_change":"2024-09-18T01:00:51.958851+0000","last_active":"2024-09-18T01:01:15.340790+0000","last_peered":"2024-09-18T01:01:15.340790+0000","last_clean":"2024-09-18T01:01:15.340790+0000","last_became_active":"2024-09-18T01:00:51.953781+0000","last_became_peered":"2024-09-18T01:00:51.953781+0000","last_unstale":"2024-09-18T01:01:15.340790+0000","last_undegraded":"2024-09-18T01:01:15.340790+0000","last_fullsized":"2024-09-18T01:01:15.340790+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:44.107525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001459178,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":30614,"num_read_kb":78260,"num_write":886,"num_write_kb":27079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":40291284,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"186'656","reported_seq":4017,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.591127+0000","last_change":"2024-09-18T01:00:44.157985+0000","last_active":"2024-09-18T01:01:15.591127+0000","last_peered":"2024-09-18T01:01:15.591127+0000","last_clean":"2024-09-18T01:01:15.591127+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T01:01:15.591127+0000","last_undegraded":"2024-09-18T01:01:15.591127+0000","last_fullsized":"2024-09-18T01:01:15.591127+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:27.075227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039694869999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4869,"num_read_kb":13814,"num_write":877,"num_write_kb":42194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"185'676","reported_seq":7355,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590783+0000","last_change":"2024-09-18T01:00:44.156705+0000","last_active":"2024-09-18T01:01:15.590783+0000","last_peered":"2024-09-18T01:01:15.590783+0000","last_clean":"2024-09-18T01:01:15.590783+0000","last_became_active":"2024-09-18T01:00:20.017711+0000","last_became_peered":"2024-09-18T01:00:20.017711+0000","last_unstale":"2024-09-18T01:01:15.590783+0000","last_undegraded":"2024-09-18T01:01:15.590783+0000","last_fullsized":"2024-09-18T01:01:15.590783+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:58:49.265670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028584439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7284,"num_read_kb":46982,"num_write":773,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":13346337,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"186'641","reported_seq":8980,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380983+0000","last_change":"2024-09-18T01:01:01.955368+0000","last_active":"2024-09-18T01:01:14.380983+0000","last_peered":"2024-09-18T01:01:14.380983+0000","last_clean":"2024-09-18T01:01:14.380983+0000","last_became_active":"2024-09-18T01:01:01.952512+0000","last_became_peered":"2024-09-18T01:01:01.952512+0000","last_unstale":"2024-09-18T01:01:14.380983+0000","last_undegraded":"2024-09-18T01:01:14.380983+0000","last_fullsized":"2024-09-18T01:01:14.380983+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:36:14.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887849999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8507,"num_read_kb":37252,"num_write":699,"num_write_kb":22150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":946,"num_bytes_recovered":37601010,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"184'519","reported_seq":3007,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941255+0000","last_change":"2024-09-18T01:01:02.959624+0000","last_active":"2024-09-18T01:01:14.941255+0000","last_peered":"2024-09-18T01:01:14.941255+0000","last_clean":"2024-09-18T01:01:14.941255+0000","last_became_active":"2024-09-18T01:01:02.956655+0000","last_became_peered":"2024-09-18T01:01:02.956655+0000","last_unstale":"2024-09-18T01:01:14.941255+0000","last_undegraded":"2024-09-18T01:01:14.941255+0000","last_fullsized":"2024-09-18T01:01:14.941255+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:53:56.284343+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010838409,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2322,"num_read_kb":15300,"num_write":809,"num_write_kb":48070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":22291504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"181'371","reported_seq":3210,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381490+0000","last_change":"2024-09-18T01:01:01.948935+0000","last_active":"2024-09-18T01:01:14.381490+0000","last_peered":"2024-09-18T01:01:14.381490+0000","last_clean":"2024-09-18T01:01:14.381490+0000","last_became_active":"2024-09-18T01:01:01.947158+0000","last_became_peered":"2024-09-18T01:01:01.947158+0000","last_unstale":"2024-09-18T01:01:14.381490+0000","last_undegraded":"2024-09-18T01:01:14.381490+0000","last_fullsized":"2024-09-18T01:01:14.381490+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:05.169088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013361650000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2452,"num_read_kb":3816,"num_write":339,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":3474495,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":127,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382692+0000","last_change":"2024-09-18T01:00:59.238970+0000","last_active":"2024-09-18T01:01:14.382692+0000","last_peered":"2024-09-18T01:01:14.382692+0000","last_clean":"2024-09-18T01:01:14.382692+0000","last_became_active":"2024-09-18T01:00:59.238106+0000","last_became_peered":"2024-09-18T01:00:59.238106+0000","last_unstale":"2024-09-18T01:01:14.382692+0000","last_undegraded":"2024-09-18T01:01:14.382692+0000","last_fullsized":"2024-09-18T01:01:14.382692+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:15:05.506655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"186'777","reported_seq":4883,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.341002+0000","last_change":"2024-09-18T01:00:44.148713+0000","last_active":"2024-09-18T01:01:15.341002+0000","last_peered":"2024-09-18T01:01:15.341002+0000","last_clean":"2024-09-18T01:01:15.341002+0000","last_became_active":"2024-09-18T01:00:11.936218+0000","last_became_peered":"2024-09-18T01:00:11.936218+0000","last_unstale":"2024-09-18T01:01:15.341002+0000","last_undegraded":"2024-09-18T01:01:15.341002+0000","last_fullsized":"2024-09-18T01:01:15.341002+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:24.939297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002351681,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5488,"num_read_kb":22949,"num_write":1002,"num_write_kb":4650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":2805113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":412,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340720+0000","last_change":"2024-09-18T01:00:59.233973+0000","last_active":"2024-09-18T01:01:15.340720+0000","last_peered":"2024-09-18T01:01:15.340720+0000","last_clean":"2024-09-18T01:01:15.340720+0000","last_became_active":"2024-09-18T01:00:59.231756+0000","last_became_peered":"2024-09-18T01:00:59.231756+0000","last_unstale":"2024-09-18T01:01:15.340720+0000","last_undegraded":"2024-09-18T01:01:15.340720+0000","last_fullsized":"2024-09-18T01:01:15.340720+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:50:54.853680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"185'506","reported_seq":8883,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.341145+0000","last_change":"2024-09-18T01:00:44.150561+0000","last_active":"2024-09-18T01:01:15.341145+0000","last_peered":"2024-09-18T01:01:15.341145+0000","last_clean":"2024-09-18T01:01:15.341145+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T01:01:15.341145+0000","last_undegraded":"2024-09-18T01:01:15.341145+0000","last_fullsized":"2024-09-18T01:01:15.341145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:44.745709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036716330000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8918,"num_read_kb":44936,"num_write":410,"num_write_kb":13864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382260+0000","last_change":"2024-09-18T01:00:15.991240+0000","last_active":"2024-09-18T01:01:14.382260+0000","last_peered":"2024-09-18T01:01:14.382260+0000","last_clean":"2024-09-18T01:01:14.382260+0000","last_became_active":"2024-09-18T01:00:15.990930+0000","last_became_peered":"2024-09-18T01:00:15.990930+0000","last_unstale":"2024-09-18T01:01:14.382260+0000","last_undegraded":"2024-09-18T01:01:14.382260+0000","last_fullsized":"2024-09-18T01:01:14.382260+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:55.616429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"185'348","reported_seq":4491,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173966+0000","last_change":"2024-09-18T01:00:44.260854+0000","last_active":"2024-09-18T01:01:15.173966+0000","last_peered":"2024-09-18T01:01:15.173966+0000","last_clean":"2024-09-18T01:01:15.173966+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T01:01:15.173966+0000","last_undegraded":"2024-09-18T01:01:15.173966+0000","last_fullsized":"2024-09-18T01:01:15.173966+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:09.012851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019390989999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3644,"num_read_kb":27224,"num_write":289,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":399,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590664+0000","last_change":"2024-09-18T00:57:29.339600+0000","last_active":"2024-09-18T01:01:15.590664+0000","last_peered":"2024-09-18T01:01:15.590664+0000","last_clean":"2024-09-18T01:01:15.590664+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T01:01:15.590664+0000","last_undegraded":"2024-09-18T01:01:15.590664+0000","last_fullsized":"2024-09-18T01:01:15.590664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:00.881072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"185'580","reported_seq":4494,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380809+0000","last_change":"2024-09-18T01:01:01.242483+0000","last_active":"2024-09-18T01:01:14.380809+0000","last_peered":"2024-09-18T01:01:14.380809+0000","last_clean":"2024-09-18T01:01:14.380809+0000","last_became_active":"2024-09-18T01:01:01.239957+0000","last_became_peered":"2024-09-18T01:01:01.239957+0000","last_unstale":"2024-09-18T01:01:14.380809+0000","last_undegraded":"2024-09-18T01:01:14.380809+0000","last_fullsized":"2024-09-18T01:01:14.380809+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:46.781823+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.024809728,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4830,"num_read_kb":33562,"num_write":800,"num_write_kb":24925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"186'946","reported_seq":6143,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941033+0000","last_change":"2024-09-18T01:01:03.958936+0000","last_active":"2024-09-18T01:01:14.941033+0000","last_peered":"2024-09-18T01:01:14.941033+0000","last_clean":"2024-09-18T01:01:14.941033+0000","last_became_active":"2024-09-18T01:01:03.955764+0000","last_became_peered":"2024-09-18T01:01:03.955764+0000","last_unstale":"2024-09-18T01:01:14.941033+0000","last_undegraded":"2024-09-18T01:01:14.941033+0000","last_fullsized":"2024-09-18T01:01:14.941033+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:25.024002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022401439999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7644,"num_read_kb":19178,"num_write":1076,"num_write_kb":33944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":99850853,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":56,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380143+0000","last_change":"2024-09-18T01:00:51.188692+0000","last_active":"2024-09-18T01:01:14.380143+0000","last_peered":"2024-09-18T01:01:14.380143+0000","last_clean":"2024-09-18T01:01:14.380143+0000","last_became_active":"2024-09-18T01:00:51.183713+0000","last_became_peered":"2024-09-18T01:00:51.183713+0000","last_unstale":"2024-09-18T01:01:14.380143+0000","last_undegraded":"2024-09-18T01:01:14.380143+0000","last_fullsized":"2024-09-18T01:01:14.380143+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:20:22.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"185'657","reported_seq":8382,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.590875+0000","last_change":"2024-09-18T01:00:44.156905+0000","last_active":"2024-09-18T01:01:15.590875+0000","last_peered":"2024-09-18T01:01:15.590875+0000","last_clean":"2024-09-18T01:01:15.590875+0000","last_became_active":"2024-09-18T01:00:08.951992+0000","last_became_peered":"2024-09-18T01:00:08.951992+0000","last_unstale":"2024-09-18T01:01:15.590875+0000","last_undegraded":"2024-09-18T01:01:15.590875+0000","last_fullsized":"2024-09-18T01:01:15.590875+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:59.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030555019999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7663,"num_read_kb":29924,"num_write":748,"num_write_kb":19563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":15190015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381997+0000","last_change":"2024-09-18T01:00:04.976725+0000","last_active":"2024-09-18T01:01:14.381997+0000","last_peered":"2024-09-18T01:01:14.381997+0000","last_clean":"2024-09-18T01:01:14.381997+0000","last_became_active":"2024-09-18T01:00:04.976360+0000","last_became_peered":"2024-09-18T01:00:04.976360+0000","last_unstale":"2024-09-18T01:01:14.381997+0000","last_undegraded":"2024-09-18T01:01:14.381997+0000","last_fullsized":"2024-09-18T01:01:14.381997+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:11.159126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"184'548","reported_seq":9961,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173598+0000","last_change":"2024-09-18T01:01:01.957077+0000","last_active":"2024-09-18T01:01:15.173598+0000","last_peered":"2024-09-18T01:01:15.173598+0000","last_clean":"2024-09-18T01:01:15.173598+0000","last_became_active":"2024-09-18T01:01:01.953404+0000","last_became_peered":"2024-09-18T01:01:01.953404+0000","last_unstale":"2024-09-18T01:01:15.173598+0000","last_undegraded":"2024-09-18T01:01:15.173598+0000","last_fullsized":"2024-09-18T01:01:15.173598+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:03.050834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022726180000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8919,"num_read_kb":48544,"num_write":559,"num_write_kb":18757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1056,"num_bytes_recovered":19034719,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379823+0000","last_change":"2024-09-18T01:00:15.994517+0000","last_active":"2024-09-18T01:01:14.379823+0000","last_peered":"2024-09-18T01:01:14.379823+0000","last_clean":"2024-09-18T01:01:14.379823+0000","last_became_active":"2024-09-18T01:00:15.994159+0000","last_became_peered":"2024-09-18T01:00:15.994159+0000","last_unstale":"2024-09-18T01:01:14.379823+0000","last_undegraded":"2024-09-18T01:01:14.379823+0000","last_fullsized":"2024-09-18T01:01:14.379823+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:06.775713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"186'722","reported_seq":5332,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340913+0000","last_change":"2024-09-18T01:01:01.952665+0000","last_active":"2024-09-18T01:01:15.340913+0000","last_peered":"2024-09-18T01:01:15.340913+0000","last_clean":"2024-09-18T01:01:15.340913+0000","last_became_active":"2024-09-18T01:01:01.948535+0000","last_became_peered":"2024-09-18T01:01:01.948535+0000","last_unstale":"2024-09-18T01:01:15.340913+0000","last_undegraded":"2024-09-18T01:01:15.340913+0000","last_fullsized":"2024-09-18T01:01:15.340913+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:39.959837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00139405,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5240,"num_read_kb":16439,"num_write":1146,"num_write_kb":52135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":12961092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":153,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380940+0000","last_change":"2024-09-18T01:00:58.232332+0000","last_active":"2024-09-18T01:01:14.380940+0000","last_peered":"2024-09-18T01:01:14.380940+0000","last_clean":"2024-09-18T01:01:14.380940+0000","last_became_active":"2024-09-18T01:00:58.229581+0000","last_became_peered":"2024-09-18T01:00:58.229581+0000","last_unstale":"2024-09-18T01:01:14.380940+0000","last_undegraded":"2024-09-18T01:01:14.380940+0000","last_fullsized":"2024-09-18T01:01:14.380940+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:26.291871+0000","last_clean_scrub_stamp":"2024-09-18T00:57:26.291871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:05:38.818642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":551,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912328+0000","last_change":"2024-09-18T00:57:23.478203+0000","last_active":"2024-09-18T01:01:14.912328+0000","last_peered":"2024-09-18T01:01:14.912328+0000","last_clean":"2024-09-18T01:01:14.912328+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T01:01:14.912328+0000","last_undegraded":"2024-09-18T01:01:14.912328+0000","last_fullsized":"2024-09-18T01:01:14.912328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:22.264242+0000","last_clean_scrub_stamp":"2024-09-18T00:57:22.264242+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:53.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"184'448","reported_seq":5879,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.340789+0000","last_change":"2024-09-18T01:00:44.149901+0000","last_active":"2024-09-18T01:01:15.340789+0000","last_peered":"2024-09-18T01:01:15.340789+0000","last_clean":"2024-09-18T01:01:15.340789+0000","last_became_active":"2024-09-18T01:00:08.949384+0000","last_became_peered":"2024-09-18T01:00:08.949384+0000","last_unstale":"2024-09-18T01:01:15.340789+0000","last_undegraded":"2024-09-18T01:01:15.340789+0000","last_fullsized":"2024-09-18T01:01:15.340789+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:20.064612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033952990000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5722,"num_read_kb":51790,"num_write":458,"num_write_kb":39546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":32435091,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"185'408","reported_seq":3782,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.379828+0000","last_change":"2024-09-18T01:00:44.145255+0000","last_active":"2024-09-18T01:01:14.379828+0000","last_peered":"2024-09-18T01:01:14.379828+0000","last_clean":"2024-09-18T01:01:14.379828+0000","last_became_active":"2024-09-18T01:00:18.997710+0000","last_became_peered":"2024-09-18T01:00:18.997710+0000","last_unstale":"2024-09-18T01:01:14.379828+0000","last_undegraded":"2024-09-18T01:01:14.379828+0000","last_fullsized":"2024-09-18T01:01:14.379828+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:36.454597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016382809999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3625,"num_read_kb":11553,"num_write":342,"num_write_kb":23969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":27148173,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"185'607","reported_seq":5628,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380626+0000","last_change":"2024-09-18T01:00:44.146011+0000","last_active":"2024-09-18T01:01:14.380626+0000","last_peered":"2024-09-18T01:01:14.380626+0000","last_clean":"2024-09-18T01:01:14.380626+0000","last_became_active":"2024-09-18T01:00:22.938344+0000","last_became_peered":"2024-09-18T01:00:22.938344+0000","last_unstale":"2024-09-18T01:01:14.380626+0000","last_undegraded":"2024-09-18T01:01:14.380626+0000","last_fullsized":"2024-09-18T01:01:14.380626+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:59:54.570595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021924589999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4754,"num_read_kb":46819,"num_write":596,"num_write_kb":17196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":6639875,"num_keys_recovered":85,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"185'515","reported_seq":5914,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.382003+0000","last_change":"2024-09-18T01:00:44.262988+0000","last_active":"2024-09-18T01:01:14.382003+0000","last_peered":"2024-09-18T01:01:14.382003+0000","last_clean":"2024-09-18T01:01:14.382003+0000","last_became_active":"2024-09-18T01:00:10.954910+0000","last_became_peered":"2024-09-18T01:00:10.954910+0000","last_unstale":"2024-09-18T01:01:14.382003+0000","last_undegraded":"2024-09-18T01:01:14.382003+0000","last_fullsized":"2024-09-18T01:01:14.382003+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:45.069175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042875949999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5466,"num_read_kb":66445,"num_write":496,"num_write_kb":15031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15399944,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"185'566","reported_seq":7107,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941155+0000","last_change":"2024-09-18T01:01:02.956425+0000","last_active":"2024-09-18T01:01:14.941155+0000","last_peered":"2024-09-18T01:01:14.941155+0000","last_clean":"2024-09-18T01:01:14.941155+0000","last_became_active":"2024-09-18T01:01:02.953950+0000","last_became_peered":"2024-09-18T01:01:02.953950+0000","last_unstale":"2024-09-18T01:01:14.941155+0000","last_undegraded":"2024-09-18T01:01:14.941155+0000","last_fullsized":"2024-09-18T01:01:14.941155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:11.064860+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0026394880000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6401,"num_read_kb":47394,"num_write":633,"num_write_kb":22377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":560,"num_bytes_recovered":26783458,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"184'976","reported_seq":6804,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381748+0000","last_change":"2024-09-18T01:00:44.146438+0000","last_active":"2024-09-18T01:01:14.381748+0000","last_peered":"2024-09-18T01:01:14.381748+0000","last_clean":"2024-09-18T01:01:14.381748+0000","last_became_active":"2024-09-18T01:00:21.923279+0000","last_became_peered":"2024-09-18T01:00:21.923279+0000","last_unstale":"2024-09-18T01:01:14.381748+0000","last_undegraded":"2024-09-18T01:01:14.381748+0000","last_fullsized":"2024-09-18T01:01:14.381748+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:23.093467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026100659999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8636,"num_read_kb":40955,"num_write":1143,"num_write_kb":30743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":80305919,"num_keys_recovered":163,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"183'713","reported_seq":4745,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380694+0000","last_change":"2024-09-18T01:01:02.959668+0000","last_active":"2024-09-18T01:01:14.380694+0000","last_peered":"2024-09-18T01:01:14.380694+0000","last_clean":"2024-09-18T01:01:14.380694+0000","last_became_active":"2024-09-18T01:01:02.954337+0000","last_became_peered":"2024-09-18T01:01:02.954337+0000","last_unstale":"2024-09-18T01:01:14.380694+0000","last_undegraded":"2024-09-18T01:01:14.380694+0000","last_fullsized":"2024-09-18T01:01:14.380694+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:56:25.045514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0093305890000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4741,"num_read_kb":29100,"num_write":948,"num_write_kb":26466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":41587165,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"184'563","reported_seq":4380,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.941359+0000","last_change":"2024-09-18T01:01:01.950143+0000","last_active":"2024-09-18T01:01:14.941359+0000","last_peered":"2024-09-18T01:01:14.941359+0000","last_clean":"2024-09-18T01:01:14.941359+0000","last_became_active":"2024-09-18T01:01:01.948267+0000","last_became_peered":"2024-09-18T01:01:01.948267+0000","last_unstale":"2024-09-18T01:01:14.941359+0000","last_undegraded":"2024-09-18T01:01:14.941359+0000","last_fullsized":"2024-09-18T01:01:14.941359+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:43:18.359056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024190230000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5311,"num_read_kb":28191,"num_write":598,"num_write_kb":9287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6320640,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"184'456","reported_seq":5302,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.912243+0000","last_change":"2024-09-18T01:00:44.148986+0000","last_active":"2024-09-18T01:01:14.912243+0000","last_peered":"2024-09-18T01:01:14.912243+0000","last_clean":"2024-09-18T01:01:14.912243+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T01:01:14.912243+0000","last_undegraded":"2024-09-18T01:01:14.912243+0000","last_fullsized":"2024-09-18T01:01:14.912243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:48:20.609462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001443239,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":51584,"num_write":357,"num_write_kb":26802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"183'292","reported_seq":3444,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.380096+0000","last_change":"2024-09-18T01:01:02.956466+0000","last_active":"2024-09-18T01:01:14.380096+0000","last_peered":"2024-09-18T01:01:14.380096+0000","last_clean":"2024-09-18T01:01:14.380096+0000","last_became_active":"2024-09-18T01:01:02.950361+0000","last_became_peered":"2024-09-18T01:01:02.950361+0000","last_unstale":"2024-09-18T01:01:14.380096+0000","last_undegraded":"2024-09-18T01:01:14.380096+0000","last_fullsized":"2024-09-18T01:01:14.380096+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:56:04.680353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033253200000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1897,"num_read_kb":13175,"num_write":246,"num_write_kb":14260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":15824068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"184'638","reported_seq":6909,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:15.173643+0000","last_change":"2024-09-18T01:00:44.261997+0000","last_active":"2024-09-18T01:01:15.173643+0000","last_peered":"2024-09-18T01:01:15.173643+0000","last_clean":"2024-09-18T01:01:15.173643+0000","last_became_active":"2024-09-18T01:00:21.926814+0000","last_became_peered":"2024-09-18T01:00:21.926814+0000","last_unstale":"2024-09-18T01:01:15.173643+0000","last_undegraded":"2024-09-18T01:01:15.173643+0000","last_fullsized":"2024-09-18T01:01:15.173643+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:04.145432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002915932,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6954,"num_read_kb":17130,"num_write":754,"num_write_kb":11285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":32804395,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"185'521","reported_seq":4837,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:14.381219+0000","last_change":"2024-09-18T01:01:01.245238+0000","last_active":"2024-09-18T01:01:14.381219+0000","last_peered":"2024-09-18T01:01:14.381219+0000","last_clean":"2024-09-18T01:01:14.381219+0000","last_became_active":"2024-09-18T01:01:01.242541+0000","last_became_peered":"2024-09-18T01:01:01.242541+0000","last_unstale":"2024-09-18T01:01:14.381219+0000","last_undegraded":"2024-09-18T01:01:14.381219+0000","last_fullsized":"2024-09-18T01:01:14.381219+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:57:36.358539+0000","last_clean_scrub_stamp":"2024-09-18T00:57:36.358539+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:44.647014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027658140000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5114,"num_read_kb":26799,"num_write":585,"num_write_kb":11735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":9379374,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":482,"num_read":220594,"num_read_kb":1067291,"num_write":22261,"num_write_kb":688577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12013,"num_bytes_recovered":693846180,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19538,"ondisk_log_size":19538,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47600,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324240,"statfs":{"total":96636764160,"available":96588021760,"internally_reserved":0,"allocated":2584576,"data_stored":1243311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52344,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319496,"statfs":{"total":96636764160,"available":96583163904,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509494,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60080,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311760,"statfs":{"total":96636764160,"available":96575242240,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55920,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315920,"statfs":{"total":96636764160,"available":96579502080,"internally_reserved":0,"allocated":2584576,"data_stored":1243311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60080,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311760,"statfs":{"total":96636764160,"available":96575242240,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52344,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319496,"statfs":{"total":96636764160,"available":96583163904,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47600,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324240,"statfs":{"total":96636764160,"available":96588021760,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51752,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320088,"statfs":{"total":96636764160,"available":96583770112,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:01:24.982 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.982 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.982 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.982 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.982 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.983 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.983 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.983 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.983 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.983 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.983 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.983 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.983 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-18T00:57:26.291871+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=26, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.984 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.984 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-18T00:57:26.291871+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=26, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.984 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.984 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-18T00:57:26.291871+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=26, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.984 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.984 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-18T00:57:26.291871+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=26, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.985 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.985 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.985 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-18T00:57:26.291871+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=26, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.985 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.985 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-18T00:57:26.291871+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=26, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.985 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.985 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-18T00:57:26.291871+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=26, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.985 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.985 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-18T00:57:26.291871+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=26, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.986 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-18T00:57:22.264242+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=22, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.986 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.986 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.986 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.986 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.986 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.986 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.986 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.986 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.987 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.987 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.987 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.987 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-18T00:57:36.358539+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=57, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=1, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:01:24.987 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-18T01:01:44.989 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:01:45.313 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-18T01:01:45.314 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-18T01:01:45.330 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-18T01:01:43.954576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":482,"num_read":220710,"num_read_kb":1067391,"num_write":22422,"num_write_kb":690995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12021,"num_bytes_recovered":693846218,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20148,"ondisk_log_size":20148,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":431832,"kb_used_data":21336,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754542888,"statfs":{"total":773094113280,"available":772651917312,"internally_reserved":0,"allocated":21848064,"data_stored":11127148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003971"},"pg_stats":[{"pgid":"3.1f","version":"214'637","reported_seq":4248,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:24.987081+0000","last_change":"2024-09-18T01:01:24.987263+0000","last_active":"2024-09-18T01:01:24.987081+0000","last_peered":"2024-09-18T01:01:24.987081+0000","last_clean":"2024-09-18T01:01:24.987081+0000","last_became_active":"2024-09-18T00:57:36.900198+0000","last_became_peered":"2024-09-18T00:57:36.900198+0000","last_unstale":"2024-09-18T01:01:24.987081+0000","last_undegraded":"2024-09-18T01:01:24.987081+0000","last_fullsized":"2024-09-18T01:01:24.987081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'637","last_scrub_stamp":"2024-09-18T01:01:24.986839+0000","last_deep_scrub":"214'637","last_deep_scrub_stamp":"2024-09-18T01:01:24.986839+0000","last_clean_scrub_stamp":"2024-09-18T01:01:24.986839+0000","objects_scrubbed":16,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:54.069218+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.001427074,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6437,"num_read_kb":23165,"num_write":614,"num_write_kb":11006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"214'668","reported_seq":5791,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:28.990762+0000","last_change":"2024-09-18T01:01:28.990762+0000","last_active":"2024-09-18T01:01:28.990762+0000","last_peered":"2024-09-18T01:01:28.990762+0000","last_clean":"2024-09-18T01:01:28.990762+0000","last_became_active":"2024-09-18T01:01:01.245192+0000","last_became_peered":"2024-09-18T01:01:01.245192+0000","last_unstale":"2024-09-18T01:01:28.990762+0000","last_undegraded":"2024-09-18T01:01:28.990762+0000","last_fullsized":"2024-09-18T01:01:28.990762+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'668","last_scrub_stamp":"2024-09-18T01:01:28.990437+0000","last_deep_scrub":"214'668","last_deep_scrub_stamp":"2024-09-18T01:01:28.990437+0000","last_clean_scrub_stamp":"2024-09-18T01:01:28.990437+0000","objects_scrubbed":22,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:22.666464+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0041620060000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4822,"num_read_kb":36392,"num_write":703,"num_write_kb":11559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":30825045,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"214'819","reported_seq":12999,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:34.331014+0000","last_change":"2024-09-18T01:01:34.331122+0000","last_active":"2024-09-18T01:01:34.331014+0000","last_peered":"2024-09-18T01:01:34.331014+0000","last_clean":"2024-09-18T01:01:34.331014+0000","last_became_active":"2024-09-18T01:01:03.956061+0000","last_became_peered":"2024-09-18T01:01:03.956061+0000","last_unstale":"2024-09-18T01:01:34.331014+0000","last_undegraded":"2024-09-18T01:01:34.331014+0000","last_fullsized":"2024-09-18T01:01:34.331014+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'819","last_scrub_stamp":"2024-09-18T01:01:34.330822+0000","last_deep_scrub":"214'819","last_deep_scrub_stamp":"2024-09-18T01:01:34.330822+0000","last_clean_scrub_stamp":"2024-09-18T01:01:34.330822+0000","objects_scrubbed":19,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:38:45.644157+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.0034116569999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":13334,"num_read_kb":40113,"num_write":945,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":705,"num_bytes_recovered":22094834,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"214'680","reported_seq":4863,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:24.875863+0000","last_change":"2024-09-18T01:01:24.875974+0000","last_active":"2024-09-18T01:01:24.875863+0000","last_peered":"2024-09-18T01:01:24.875863+0000","last_clean":"2024-09-18T01:01:24.875863+0000","last_became_active":"2024-09-18T01:01:01.945747+0000","last_became_peered":"2024-09-18T01:01:01.945747+0000","last_unstale":"2024-09-18T01:01:24.875863+0000","last_undegraded":"2024-09-18T01:01:24.875863+0000","last_fullsized":"2024-09-18T01:01:24.875863+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'680","last_scrub_stamp":"2024-09-18T01:01:24.875668+0000","last_deep_scrub":"214'680","last_deep_scrub_stamp":"2024-09-18T01:01:24.875668+0000","last_clean_scrub_stamp":"2024-09-18T01:01:24.875668+0000","objects_scrubbed":17,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:21:20.955359+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.002343262,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5693,"num_read_kb":28138,"num_write":736,"num_write_kb":11405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":5428683,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"214'1103","reported_seq":10895,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:29.322805+0000","last_change":"2024-09-18T01:01:29.322983+0000","last_active":"2024-09-18T01:01:29.322805+0000","last_peered":"2024-09-18T01:01:29.322805+0000","last_clean":"2024-09-18T01:01:29.322805+0000","last_became_active":"2024-09-18T01:01:01.947957+0000","last_became_peered":"2024-09-18T01:01:01.947957+0000","last_unstale":"2024-09-18T01:01:29.322805+0000","last_undegraded":"2024-09-18T01:01:29.322805+0000","last_fullsized":"2024-09-18T01:01:29.322805+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'1103","last_scrub_stamp":"2024-09-18T01:01:29.322584+0000","last_deep_scrub":"214'1103","last_deep_scrub_stamp":"2024-09-18T01:01:29.322584+0000","last_clean_scrub_stamp":"2024-09-18T01:01:29.322584+0000","objects_scrubbed":10,"log_size":1103,"log_dups_size":0,"ondisk_log_size":1103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:28:36.480878+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.003279527,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12686,"num_read_kb":40787,"num_write":1375,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":629,"num_bytes_recovered":59340116,"num_keys_recovered":58,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"214'379","reported_seq":5168,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:22.796903+0000","last_change":"2024-09-18T01:01:22.796954+0000","last_active":"2024-09-18T01:01:22.796903+0000","last_peered":"2024-09-18T01:01:22.796903+0000","last_clean":"2024-09-18T01:01:22.796903+0000","last_became_active":"2024-09-18T00:57:36.897907+0000","last_became_peered":"2024-09-18T00:57:36.897907+0000","last_unstale":"2024-09-18T01:01:22.796903+0000","last_undegraded":"2024-09-18T01:01:22.796903+0000","last_fullsized":"2024-09-18T01:01:22.796903+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'379","last_scrub_stamp":"2024-09-18T01:01:22.796836+0000","last_deep_scrub":"214'379","last_deep_scrub_stamp":"2024-09-18T01:01:22.796836+0000","last_clean_scrub_stamp":"2024-09-18T01:01:22.796836+0000","objects_scrubbed":15,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:57:28.215496+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0017298630000000001,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":5410,"num_read_kb":29581,"num_write":319,"num_write_kb":12768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"214'718","reported_seq":30914,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:26.372541+0000","last_change":"2024-09-18T01:01:26.372620+0000","last_active":"2024-09-18T01:01:26.372541+0000","last_peered":"2024-09-18T01:01:26.372541+0000","last_clean":"2024-09-18T01:01:26.372541+0000","last_became_active":"2024-09-18T01:00:51.953781+0000","last_became_peered":"2024-09-18T01:00:51.953781+0000","last_unstale":"2024-09-18T01:01:26.372541+0000","last_undegraded":"2024-09-18T01:01:26.372541+0000","last_fullsized":"2024-09-18T01:01:26.372541+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'718","last_scrub_stamp":"2024-09-18T01:01:26.372397+0000","last_deep_scrub":"214'718","last_deep_scrub_stamp":"2024-09-18T01:01:26.372397+0000","last_clean_scrub_stamp":"2024-09-18T01:01:26.372397+0000","objects_scrubbed":17,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:51.487436+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.001459178,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":30614,"num_read_kb":78260,"num_write":886,"num_write_kb":27079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":40291284,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"214'674","reported_seq":4044,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:25.971286+0000","last_change":"2024-09-18T01:01:25.971392+0000","last_active":"2024-09-18T01:01:25.971286+0000","last_peered":"2024-09-18T01:01:25.971286+0000","last_clean":"2024-09-18T01:01:25.971286+0000","last_became_active":"2024-09-18T00:57:36.903613+0000","last_became_peered":"2024-09-18T00:57:36.903613+0000","last_unstale":"2024-09-18T01:01:25.971286+0000","last_undegraded":"2024-09-18T01:01:25.971286+0000","last_fullsized":"2024-09-18T01:01:25.971286+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'674","last_scrub_stamp":"2024-09-18T01:01:25.971095+0000","last_deep_scrub":"214'674","last_deep_scrub_stamp":"2024-09-18T01:01:25.971095+0000","last_clean_scrub_stamp":"2024-09-18T01:01:25.971095+0000","objects_scrubbed":18,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:48.002864+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0039694869999999998,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4869,"num_read_kb":13814,"num_write":877,"num_write_kb":42194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"214'686","reported_seq":7374,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:27.969360+0000","last_change":"2024-09-18T01:01:27.969452+0000","last_active":"2024-09-18T01:01:27.969360+0000","last_peered":"2024-09-18T01:01:27.969360+0000","last_clean":"2024-09-18T01:01:27.969360+0000","last_became_active":"2024-09-18T01:00:20.017711+0000","last_became_peered":"2024-09-18T01:00:20.017711+0000","last_unstale":"2024-09-18T01:01:27.969360+0000","last_undegraded":"2024-09-18T01:01:27.969360+0000","last_fullsized":"2024-09-18T01:01:27.969360+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'686","last_scrub_stamp":"2024-09-18T01:01:27.969180+0000","last_deep_scrub":"214'686","last_deep_scrub_stamp":"2024-09-18T01:01:27.969180+0000","last_clean_scrub_stamp":"2024-09-18T01:01:27.969180+0000","objects_scrubbed":10,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:37.838800+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0028584439999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7284,"num_read_kb":46982,"num_write":773,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":13346337,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"214'651","reported_seq":8999,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:23.473919+0000","last_change":"2024-09-18T01:01:23.473958+0000","last_active":"2024-09-18T01:01:23.473919+0000","last_peered":"2024-09-18T01:01:23.473919+0000","last_clean":"2024-09-18T01:01:23.473919+0000","last_became_active":"2024-09-18T01:01:01.952512+0000","last_became_peered":"2024-09-18T01:01:01.952512+0000","last_unstale":"2024-09-18T01:01:23.473919+0000","last_undegraded":"2024-09-18T01:01:23.473919+0000","last_fullsized":"2024-09-18T01:01:23.473919+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'651","last_scrub_stamp":"2024-09-18T01:01:23.473849+0000","last_deep_scrub":"214'651","last_deep_scrub_stamp":"2024-09-18T01:01:23.473849+0000","last_clean_scrub_stamp":"2024-09-18T01:01:23.473849+0000","objects_scrubbed":10,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:43:29.711963+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0016887849999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8507,"num_read_kb":37252,"num_write":699,"num_write_kb":22150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":946,"num_bytes_recovered":37601010,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"214'538","reported_seq":3035,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:23.379475+0000","last_change":"2024-09-18T01:01:23.379574+0000","last_active":"2024-09-18T01:01:23.379475+0000","last_peered":"2024-09-18T01:01:23.379475+0000","last_clean":"2024-09-18T01:01:23.379475+0000","last_became_active":"2024-09-18T01:01:02.956655+0000","last_became_peered":"2024-09-18T01:01:02.956655+0000","last_unstale":"2024-09-18T01:01:23.379475+0000","last_undegraded":"2024-09-18T01:01:23.379475+0000","last_fullsized":"2024-09-18T01:01:23.379475+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'538","last_scrub_stamp":"2024-09-18T01:01:23.379392+0000","last_deep_scrub":"214'538","last_deep_scrub_stamp":"2024-09-18T01:01:23.379392+0000","last_clean_scrub_stamp":"2024-09-18T01:01:23.379392+0000","objects_scrubbed":19,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:10.541137+0000","scrub_duration":31,"objects_trimmed":2,"snaptrim_duration":0.010838409,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":2322,"num_read_kb":15300,"num_write":809,"num_write_kb":48070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":22291504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"214'385","reported_seq":3233,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:32.303916+0000","last_change":"2024-09-18T01:01:32.304125+0000","last_active":"2024-09-18T01:01:32.303916+0000","last_peered":"2024-09-18T01:01:32.303916+0000","last_clean":"2024-09-18T01:01:32.303916+0000","last_became_active":"2024-09-18T01:01:01.947158+0000","last_became_peered":"2024-09-18T01:01:01.947158+0000","last_unstale":"2024-09-18T01:01:32.303916+0000","last_undegraded":"2024-09-18T01:01:32.303916+0000","last_fullsized":"2024-09-18T01:01:32.303916+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'385","last_scrub_stamp":"2024-09-18T01:01:32.303688+0000","last_deep_scrub":"214'385","last_deep_scrub_stamp":"2024-09-18T01:01:32.303688+0000","last_clean_scrub_stamp":"2024-09-18T01:01:32.303688+0000","objects_scrubbed":14,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:07:18.557106+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0013361650000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2452,"num_read_kb":3816,"num_write":339,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":3474495,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:30.334292+0000","last_change":"2024-09-18T01:01:30.334398+0000","last_active":"2024-09-18T01:01:30.334292+0000","last_peered":"2024-09-18T01:01:30.334292+0000","last_clean":"2024-09-18T01:01:30.334292+0000","last_became_active":"2024-09-18T01:00:59.238106+0000","last_became_peered":"2024-09-18T01:00:59.238106+0000","last_unstale":"2024-09-18T01:01:30.334292+0000","last_undegraded":"2024-09-18T01:01:30.334292+0000","last_fullsized":"2024-09-18T01:01:30.334292+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:01:30.334002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:01:30.334002+0000","last_clean_scrub_stamp":"2024-09-18T01:01:30.334002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:32:22.036718+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"214'795","reported_seq":4910,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:25.402863+0000","last_change":"2024-09-18T01:01:25.402962+0000","last_active":"2024-09-18T01:01:25.402863+0000","last_peered":"2024-09-18T01:01:25.402863+0000","last_clean":"2024-09-18T01:01:25.402863+0000","last_became_active":"2024-09-18T01:00:11.936218+0000","last_became_peered":"2024-09-18T01:00:11.936218+0000","last_unstale":"2024-09-18T01:01:25.402863+0000","last_undegraded":"2024-09-18T01:01:25.402863+0000","last_fullsized":"2024-09-18T01:01:25.402863+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'795","last_scrub_stamp":"2024-09-18T01:01:25.402676+0000","last_deep_scrub":"214'795","last_deep_scrub_stamp":"2024-09-18T01:01:25.402676+0000","last_clean_scrub_stamp":"2024-09-18T01:01:25.402676+0000","objects_scrubbed":18,"log_size":795,"log_dups_size":0,"ondisk_log_size":795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:39:28.126753+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.002351681,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5488,"num_read_kb":22949,"num_write":1002,"num_write_kb":4650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":2805113,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":421,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:27.318487+0000","last_change":"2024-09-18T01:01:27.318544+0000","last_active":"2024-09-18T01:01:27.318487+0000","last_peered":"2024-09-18T01:01:27.318487+0000","last_clean":"2024-09-18T01:01:27.318487+0000","last_became_active":"2024-09-18T01:00:59.231756+0000","last_became_peered":"2024-09-18T01:00:59.231756+0000","last_unstale":"2024-09-18T01:01:27.318487+0000","last_undegraded":"2024-09-18T01:01:27.318487+0000","last_fullsized":"2024-09-18T01:01:27.318487+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:01:27.318299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:01:27.318299+0000","last_clean_scrub_stamp":"2024-09-18T01:01:27.318299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:45:02.908005+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00092200400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"214'517","reported_seq":8903,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:23.338757+0000","last_change":"2024-09-18T01:01:23.338846+0000","last_active":"2024-09-18T01:01:23.338757+0000","last_peered":"2024-09-18T01:01:23.338757+0000","last_clean":"2024-09-18T01:01:23.338757+0000","last_became_active":"2024-09-18T00:57:36.916093+0000","last_became_peered":"2024-09-18T00:57:36.916093+0000","last_unstale":"2024-09-18T01:01:23.338757+0000","last_undegraded":"2024-09-18T01:01:23.338757+0000","last_fullsized":"2024-09-18T01:01:23.338757+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'517","last_scrub_stamp":"2024-09-18T01:01:23.338556+0000","last_deep_scrub":"214'517","last_deep_scrub_stamp":"2024-09-18T01:01:23.338556+0000","last_clean_scrub_stamp":"2024-09-18T01:01:23.338556+0000","objects_scrubbed":11,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:22:12.746048+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0036716330000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8918,"num_read_kb":44936,"num_write":410,"num_write_kb":13864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:23.234739+0000","last_change":"2024-09-18T01:01:23.234773+0000","last_active":"2024-09-18T01:01:23.234739+0000","last_peered":"2024-09-18T01:01:23.234739+0000","last_clean":"2024-09-18T01:01:23.234739+0000","last_became_active":"2024-09-18T01:00:15.990930+0000","last_became_peered":"2024-09-18T01:00:15.990930+0000","last_unstale":"2024-09-18T01:01:23.234739+0000","last_undegraded":"2024-09-18T01:01:23.234739+0000","last_fullsized":"2024-09-18T01:01:23.234739+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:01:23.234664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:01:23.234664+0000","last_clean_scrub_stamp":"2024-09-18T01:01:23.234664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:45:19.575032+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"214'366","reported_seq":4518,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:25.003965+0000","last_change":"2024-09-18T01:01:25.004072+0000","last_active":"2024-09-18T01:01:25.003965+0000","last_peered":"2024-09-18T01:01:25.003965+0000","last_clean":"2024-09-18T01:01:25.003965+0000","last_became_active":"2024-09-18T00:57:36.913213+0000","last_became_peered":"2024-09-18T00:57:36.913213+0000","last_unstale":"2024-09-18T01:01:25.003965+0000","last_undegraded":"2024-09-18T01:01:25.003965+0000","last_fullsized":"2024-09-18T01:01:25.003965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'366","last_scrub_stamp":"2024-09-18T01:01:25.003783+0000","last_deep_scrub":"214'366","last_deep_scrub_stamp":"2024-09-18T01:01:25.003783+0000","last_clean_scrub_stamp":"2024-09-18T01:01:25.003783+0000","objects_scrubbed":18,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:49:28.701930+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0019390989999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3644,"num_read_kb":27224,"num_write":289,"num_write_kb":17188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":409,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:23.970669+0000","last_change":"2024-09-18T01:01:23.970669+0000","last_active":"2024-09-18T01:01:23.970669+0000","last_peered":"2024-09-18T01:01:23.970669+0000","last_clean":"2024-09-18T01:01:23.970669+0000","last_became_active":"2024-09-18T00:57:27.410241+0000","last_became_peered":"2024-09-18T00:57:27.410241+0000","last_unstale":"2024-09-18T01:01:23.970669+0000","last_undegraded":"2024-09-18T01:01:23.970669+0000","last_fullsized":"2024-09-18T01:01:23.970669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:01:23.970292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:01:23.970292+0000","last_clean_scrub_stamp":"2024-09-18T01:01:23.970292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:53:55.589416+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010824929999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"214'597","reported_seq":4520,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:22.490375+0000","last_change":"2024-09-18T01:01:22.490429+0000","last_active":"2024-09-18T01:01:22.490375+0000","last_peered":"2024-09-18T01:01:22.490375+0000","last_clean":"2024-09-18T01:01:22.490375+0000","last_became_active":"2024-09-18T01:01:01.239957+0000","last_became_peered":"2024-09-18T01:01:01.239957+0000","last_unstale":"2024-09-18T01:01:22.490375+0000","last_undegraded":"2024-09-18T01:01:22.490375+0000","last_fullsized":"2024-09-18T01:01:22.490375+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'597","last_scrub_stamp":"2024-09-18T01:01:22.490291+0000","last_deep_scrub":"214'597","last_deep_scrub_stamp":"2024-09-18T01:01:22.490291+0000","last_clean_scrub_stamp":"2024-09-18T01:01:22.490291+0000","objects_scrubbed":17,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:01:01.385385+0000","scrub_duration":19,"objects_trimmed":2,"snaptrim_duration":0.024809728,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4830,"num_read_kb":33562,"num_write":800,"num_write_kb":24925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":14682978,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"214'963","reported_seq":6169,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:24.349953+0000","last_change":"2024-09-18T01:01:24.350010+0000","last_active":"2024-09-18T01:01:24.349953+0000","last_peered":"2024-09-18T01:01:24.349953+0000","last_clean":"2024-09-18T01:01:24.349953+0000","last_became_active":"2024-09-18T01:01:03.955764+0000","last_became_peered":"2024-09-18T01:01:03.955764+0000","last_unstale":"2024-09-18T01:01:24.349953+0000","last_undegraded":"2024-09-18T01:01:24.349953+0000","last_fullsized":"2024-09-18T01:01:24.349953+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'963","last_scrub_stamp":"2024-09-18T01:01:24.349882+0000","last_deep_scrub":"214'963","last_deep_scrub_stamp":"2024-09-18T01:01:24.349882+0000","last_clean_scrub_stamp":"2024-09-18T01:01:24.349882+0000","objects_scrubbed":17,"log_size":963,"log_dups_size":0,"ondisk_log_size":963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:40:45.116170+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0022401439999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7644,"num_read_kb":19178,"num_write":1076,"num_write_kb":33944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":99850853,"num_keys_recovered":61,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"214'3","reported_seq":66,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:28.257702+0000","last_change":"2024-09-18T01:01:28.257768+0000","last_active":"2024-09-18T01:01:28.257702+0000","last_peered":"2024-09-18T01:01:28.257702+0000","last_clean":"2024-09-18T01:01:28.257702+0000","last_became_active":"2024-09-18T01:00:51.183713+0000","last_became_peered":"2024-09-18T01:00:51.183713+0000","last_unstale":"2024-09-18T01:01:28.257702+0000","last_undegraded":"2024-09-18T01:01:28.257702+0000","last_fullsized":"2024-09-18T01:01:28.257702+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'3","last_scrub_stamp":"2024-09-18T01:01:28.257500+0000","last_deep_scrub":"214'3","last_deep_scrub_stamp":"2024-09-18T01:01:28.257500+0000","last_clean_scrub_stamp":"2024-09-18T01:01:28.257500+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:45.315691+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0013993829999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"214'673","reported_seq":8408,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:26.982764+0000","last_change":"2024-09-18T01:01:26.982764+0000","last_active":"2024-09-18T01:01:26.982764+0000","last_peered":"2024-09-18T01:01:26.982764+0000","last_clean":"2024-09-18T01:01:26.982764+0000","last_became_active":"2024-09-18T01:00:08.951992+0000","last_became_peered":"2024-09-18T01:00:08.951992+0000","last_unstale":"2024-09-18T01:01:26.982764+0000","last_undegraded":"2024-09-18T01:01:26.982764+0000","last_fullsized":"2024-09-18T01:01:26.982764+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'673","last_scrub_stamp":"2024-09-18T01:01:26.982294+0000","last_deep_scrub":"214'673","last_deep_scrub_stamp":"2024-09-18T01:01:26.982294+0000","last_clean_scrub_stamp":"2024-09-18T01:01:26.982294+0000","objects_scrubbed":16,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:02:31.939542+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0030555019999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7663,"num_read_kb":29924,"num_write":748,"num_write_kb":19563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":15190015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:24.696324+0000","last_change":"2024-09-18T01:01:24.696382+0000","last_active":"2024-09-18T01:01:24.696324+0000","last_peered":"2024-09-18T01:01:24.696324+0000","last_clean":"2024-09-18T01:01:24.696324+0000","last_became_active":"2024-09-18T01:00:04.976360+0000","last_became_peered":"2024-09-18T01:00:04.976360+0000","last_unstale":"2024-09-18T01:01:24.696324+0000","last_undegraded":"2024-09-18T01:01:24.696324+0000","last_fullsized":"2024-09-18T01:01:24.696324+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:01:24.696122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:01:24.696122+0000","last_clean_scrub_stamp":"2024-09-18T01:01:24.696122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:12:10.391767+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"214'564","reported_seq":9986,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:26.992271+0000","last_change":"2024-09-18T01:01:26.992358+0000","last_active":"2024-09-18T01:01:26.992271+0000","last_peered":"2024-09-18T01:01:26.992271+0000","last_clean":"2024-09-18T01:01:26.992271+0000","last_became_active":"2024-09-18T01:01:01.953404+0000","last_became_peered":"2024-09-18T01:01:01.953404+0000","last_unstale":"2024-09-18T01:01:26.992271+0000","last_undegraded":"2024-09-18T01:01:26.992271+0000","last_fullsized":"2024-09-18T01:01:26.992271+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'564","last_scrub_stamp":"2024-09-18T01:01:26.992095+0000","last_deep_scrub":"214'564","last_deep_scrub_stamp":"2024-09-18T01:01:26.992095+0000","last_clean_scrub_stamp":"2024-09-18T01:01:26.992095+0000","objects_scrubbed":16,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:22.448642+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0022726180000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8919,"num_read_kb":48544,"num_write":559,"num_write_kb":18757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1056,"num_bytes_recovered":19034719,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:24.283182+0000","last_change":"2024-09-18T01:01:24.283253+0000","last_active":"2024-09-18T01:01:24.283182+0000","last_peered":"2024-09-18T01:01:24.283182+0000","last_clean":"2024-09-18T01:01:24.283182+0000","last_became_active":"2024-09-18T01:00:15.994159+0000","last_became_peered":"2024-09-18T01:00:15.994159+0000","last_unstale":"2024-09-18T01:01:24.283182+0000","last_undegraded":"2024-09-18T01:01:24.283182+0000","last_fullsized":"2024-09-18T01:01:24.283182+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:01:24.282934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:01:24.282934+0000","last_clean_scrub_stamp":"2024-09-18T01:01:24.282934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:09:34.125855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"214'744","reported_seq":5363,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:28.384807+0000","last_change":"2024-09-18T01:01:28.384907+0000","last_active":"2024-09-18T01:01:28.384807+0000","last_peered":"2024-09-18T01:01:28.384807+0000","last_clean":"2024-09-18T01:01:28.384807+0000","last_became_active":"2024-09-18T01:01:01.948535+0000","last_became_peered":"2024-09-18T01:01:01.948535+0000","last_unstale":"2024-09-18T01:01:28.384807+0000","last_undegraded":"2024-09-18T01:01:28.384807+0000","last_fullsized":"2024-09-18T01:01:28.384807+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'744","last_scrub_stamp":"2024-09-18T01:01:28.384622+0000","last_deep_scrub":"214'744","last_deep_scrub_stamp":"2024-09-18T01:01:28.384622+0000","last_clean_scrub_stamp":"2024-09-18T01:01:28.384622+0000","objects_scrubbed":22,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:59:20.320101+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.00139405,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":5240,"num_read_kb":16439,"num_write":1146,"num_write_kb":52135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":12961092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"214'2","reported_seq":163,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:29.294792+0000","last_change":"2024-09-18T01:01:29.294952+0000","last_active":"2024-09-18T01:01:29.294792+0000","last_peered":"2024-09-18T01:01:29.294792+0000","last_clean":"2024-09-18T01:01:29.294792+0000","last_became_active":"2024-09-18T01:00:58.229581+0000","last_became_peered":"2024-09-18T01:00:58.229581+0000","last_unstale":"2024-09-18T01:01:29.294792+0000","last_undegraded":"2024-09-18T01:01:29.294792+0000","last_fullsized":"2024-09-18T01:01:29.294792+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'2","last_scrub_stamp":"2024-09-18T01:01:29.294535+0000","last_deep_scrub":"214'2","last_deep_scrub_stamp":"2024-09-18T01:01:29.294535+0000","last_clean_scrub_stamp":"2024-09-18T01:01:29.294535+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:31:49.747320+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0021383309999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"214'119","reported_seq":563,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:21.800806+0000","last_change":"2024-09-18T01:01:21.800806+0000","last_active":"2024-09-18T01:01:21.800806+0000","last_peered":"2024-09-18T01:01:21.800806+0000","last_clean":"2024-09-18T01:01:21.800806+0000","last_became_active":"2024-09-18T00:57:23.476224+0000","last_became_peered":"2024-09-18T00:57:23.476224+0000","last_unstale":"2024-09-18T01:01:21.800806+0000","last_undegraded":"2024-09-18T01:01:21.800806+0000","last_fullsized":"2024-09-18T01:01:21.800806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'119","last_scrub_stamp":"2024-09-18T01:01:21.800556+0000","last_deep_scrub":"214'119","last_deep_scrub_stamp":"2024-09-18T01:01:21.800556+0000","last_clean_scrub_stamp":"2024-09-18T01:01:21.800556+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:50.189887+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"214'466","reported_seq":5906,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:24.375238+0000","last_change":"2024-09-18T01:01:24.375427+0000","last_active":"2024-09-18T01:01:24.375238+0000","last_peered":"2024-09-18T01:01:24.375238+0000","last_clean":"2024-09-18T01:01:24.375238+0000","last_became_active":"2024-09-18T01:00:08.949384+0000","last_became_peered":"2024-09-18T01:00:08.949384+0000","last_unstale":"2024-09-18T01:01:24.375238+0000","last_undegraded":"2024-09-18T01:01:24.375238+0000","last_fullsized":"2024-09-18T01:01:24.375238+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'466","last_scrub_stamp":"2024-09-18T01:01:24.375021+0000","last_deep_scrub":"214'466","last_deep_scrub_stamp":"2024-09-18T01:01:24.375021+0000","last_clean_scrub_stamp":"2024-09-18T01:01:24.375021+0000","objects_scrubbed":18,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:56:32.326467+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0033952990000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5722,"num_read_kb":51790,"num_write":458,"num_write_kb":39546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":32435091,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"214'424","reported_seq":3807,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:25.347859+0000","last_change":"2024-09-18T01:01:25.348037+0000","last_active":"2024-09-18T01:01:25.347859+0000","last_peered":"2024-09-18T01:01:25.347859+0000","last_clean":"2024-09-18T01:01:25.347859+0000","last_became_active":"2024-09-18T01:00:18.997710+0000","last_became_peered":"2024-09-18T01:00:18.997710+0000","last_unstale":"2024-09-18T01:01:25.347859+0000","last_undegraded":"2024-09-18T01:01:25.347859+0000","last_fullsized":"2024-09-18T01:01:25.347859+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'424","last_scrub_stamp":"2024-09-18T01:01:25.347617+0000","last_deep_scrub":"214'424","last_deep_scrub_stamp":"2024-09-18T01:01:25.347617+0000","last_clean_scrub_stamp":"2024-09-18T01:01:25.347617+0000","objects_scrubbed":16,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:59:18.382559+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0016382809999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3625,"num_read_kb":11553,"num_write":342,"num_write_kb":23969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":27148173,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"214'626","reported_seq":5656,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:27.318583+0000","last_change":"2024-09-18T01:01:27.318698+0000","last_active":"2024-09-18T01:01:27.318583+0000","last_peered":"2024-09-18T01:01:27.318583+0000","last_clean":"2024-09-18T01:01:27.318583+0000","last_became_active":"2024-09-18T01:00:22.938344+0000","last_became_peered":"2024-09-18T01:00:22.938344+0000","last_unstale":"2024-09-18T01:01:27.318583+0000","last_undegraded":"2024-09-18T01:01:27.318583+0000","last_fullsized":"2024-09-18T01:01:27.318583+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'626","last_scrub_stamp":"2024-09-18T01:01:27.318385+0000","last_deep_scrub":"214'626","last_deep_scrub_stamp":"2024-09-18T01:01:27.318385+0000","last_clean_scrub_stamp":"2024-09-18T01:01:27.318385+0000","objects_scrubbed":19,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:33:10.840132+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.0021924589999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4754,"num_read_kb":46819,"num_write":596,"num_write_kb":17196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":6639875,"num_keys_recovered":85,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"214'527","reported_seq":5935,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:25.708406+0000","last_change":"2024-09-18T01:01:25.708495+0000","last_active":"2024-09-18T01:01:25.708406+0000","last_peered":"2024-09-18T01:01:25.708406+0000","last_clean":"2024-09-18T01:01:25.708406+0000","last_became_active":"2024-09-18T01:00:10.954910+0000","last_became_peered":"2024-09-18T01:00:10.954910+0000","last_unstale":"2024-09-18T01:01:25.708406+0000","last_undegraded":"2024-09-18T01:01:25.708406+0000","last_fullsized":"2024-09-18T01:01:25.708406+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'527","last_scrub_stamp":"2024-09-18T01:01:25.708223+0000","last_deep_scrub":"214'527","last_deep_scrub_stamp":"2024-09-18T01:01:25.708223+0000","last_clean_scrub_stamp":"2024-09-18T01:01:25.708223+0000","objects_scrubbed":12,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:25.204536+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0042875949999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5466,"num_read_kb":66445,"num_write":496,"num_write_kb":15031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":15399944,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"214'572","reported_seq":7122,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:22.312233+0000","last_change":"2024-09-18T01:01:22.312335+0000","last_active":"2024-09-18T01:01:22.312233+0000","last_peered":"2024-09-18T01:01:22.312233+0000","last_clean":"2024-09-18T01:01:22.312233+0000","last_became_active":"2024-09-18T01:01:02.953950+0000","last_became_peered":"2024-09-18T01:01:02.953950+0000","last_unstale":"2024-09-18T01:01:22.312233+0000","last_undegraded":"2024-09-18T01:01:22.312233+0000","last_fullsized":"2024-09-18T01:01:22.312233+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'572","last_scrub_stamp":"2024-09-18T01:01:22.312102+0000","last_deep_scrub":"214'572","last_deep_scrub_stamp":"2024-09-18T01:01:22.312102+0000","last_clean_scrub_stamp":"2024-09-18T01:01:22.312102+0000","objects_scrubbed":6,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:47:13.140944+0000","scrub_duration":7,"objects_trimmed":6,"snaptrim_duration":0.0026394880000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6401,"num_read_kb":47394,"num_write":633,"num_write_kb":22377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":560,"num_bytes_recovered":26783458,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"214'992","reported_seq":6829,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:26.315748+0000","last_change":"2024-09-18T01:01:26.315854+0000","last_active":"2024-09-18T01:01:26.315748+0000","last_peered":"2024-09-18T01:01:26.315748+0000","last_clean":"2024-09-18T01:01:26.315748+0000","last_became_active":"2024-09-18T01:00:21.923279+0000","last_became_peered":"2024-09-18T01:00:21.923279+0000","last_unstale":"2024-09-18T01:01:26.315748+0000","last_undegraded":"2024-09-18T01:01:26.315748+0000","last_fullsized":"2024-09-18T01:01:26.315748+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'992","last_scrub_stamp":"2024-09-18T01:01:26.315555+0000","last_deep_scrub":"214'992","last_deep_scrub_stamp":"2024-09-18T01:01:26.315555+0000","last_clean_scrub_stamp":"2024-09-18T01:01:26.315555+0000","objects_scrubbed":16,"log_size":992,"log_dups_size":0,"ondisk_log_size":992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:39:12.750346+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0026100659999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8636,"num_read_kb":40955,"num_write":1143,"num_write_kb":30743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":601,"num_bytes_recovered":80305919,"num_keys_recovered":163,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"214'726","reported_seq":4767,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:24.507141+0000","last_change":"2024-09-18T01:01:24.507187+0000","last_active":"2024-09-18T01:01:24.507141+0000","last_peered":"2024-09-18T01:01:24.507141+0000","last_clean":"2024-09-18T01:01:24.507141+0000","last_became_active":"2024-09-18T01:01:02.954337+0000","last_became_peered":"2024-09-18T01:01:02.954337+0000","last_unstale":"2024-09-18T01:01:24.507141+0000","last_undegraded":"2024-09-18T01:01:24.507141+0000","last_fullsized":"2024-09-18T01:01:24.507141+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'726","last_scrub_stamp":"2024-09-18T01:01:24.507056+0000","last_deep_scrub":"214'726","last_deep_scrub_stamp":"2024-09-18T01:01:24.507056+0000","last_clean_scrub_stamp":"2024-09-18T01:01:24.507056+0000","objects_scrubbed":13,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:02.026407+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0093305890000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":4741,"num_read_kb":29100,"num_write":948,"num_write_kb":26466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":41587165,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"214'574","reported_seq":4400,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:21.263000+0000","last_change":"2024-09-18T01:01:21.263058+0000","last_active":"2024-09-18T01:01:21.263000+0000","last_peered":"2024-09-18T01:01:21.263000+0000","last_clean":"2024-09-18T01:01:21.263000+0000","last_became_active":"2024-09-18T01:01:01.948267+0000","last_became_peered":"2024-09-18T01:01:01.948267+0000","last_unstale":"2024-09-18T01:01:21.263000+0000","last_undegraded":"2024-09-18T01:01:21.263000+0000","last_fullsized":"2024-09-18T01:01:21.263000+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'574","last_scrub_stamp":"2024-09-18T01:01:21.262896+0000","last_deep_scrub":"214'574","last_deep_scrub_stamp":"2024-09-18T01:01:21.262896+0000","last_clean_scrub_stamp":"2024-09-18T01:01:21.262896+0000","objects_scrubbed":11,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:04:47.061321+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0024190230000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5311,"num_read_kb":28191,"num_write":598,"num_write_kb":9287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6320640,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"214'475","reported_seq":5330,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:23.840844+0000","last_change":"2024-09-18T01:01:23.840904+0000","last_active":"2024-09-18T01:01:23.840844+0000","last_peered":"2024-09-18T01:01:23.840844+0000","last_clean":"2024-09-18T01:01:23.840844+0000","last_became_active":"2024-09-18T00:57:36.907300+0000","last_became_peered":"2024-09-18T00:57:36.907300+0000","last_unstale":"2024-09-18T01:01:23.840844+0000","last_undegraded":"2024-09-18T01:01:23.840844+0000","last_fullsized":"2024-09-18T01:01:23.840844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'475","last_scrub_stamp":"2024-09-18T01:01:23.840764+0000","last_deep_scrub":"214'475","last_deep_scrub_stamp":"2024-09-18T01:01:23.840764+0000","last_clean_scrub_stamp":"2024-09-18T01:01:23.840764+0000","objects_scrubbed":19,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:30:48.005752+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.001443239,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5197,"num_read_kb":51584,"num_write":357,"num_write_kb":26802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"214'302","reported_seq":3463,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:33.308384+0000","last_change":"2024-09-18T01:01:33.308466+0000","last_active":"2024-09-18T01:01:33.308384+0000","last_peered":"2024-09-18T01:01:33.308384+0000","last_clean":"2024-09-18T01:01:33.308384+0000","last_became_active":"2024-09-18T01:01:02.950361+0000","last_became_peered":"2024-09-18T01:01:02.950361+0000","last_unstale":"2024-09-18T01:01:33.308384+0000","last_undegraded":"2024-09-18T01:01:33.308384+0000","last_fullsized":"2024-09-18T01:01:33.308384+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'302","last_scrub_stamp":"2024-09-18T01:01:33.308196+0000","last_deep_scrub":"214'302","last_deep_scrub_stamp":"2024-09-18T01:01:33.308196+0000","last_clean_scrub_stamp":"2024-09-18T01:01:33.308196+0000","objects_scrubbed":10,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:35:54.427801+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0033253200000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1897,"num_read_kb":13175,"num_write":246,"num_write_kb":14260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":15824068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"214'652","reported_seq":6932,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:26.029000+0000","last_change":"2024-09-18T01:01:26.029143+0000","last_active":"2024-09-18T01:01:26.029000+0000","last_peered":"2024-09-18T01:01:26.029000+0000","last_clean":"2024-09-18T01:01:26.029000+0000","last_became_active":"2024-09-18T01:00:21.926814+0000","last_became_peered":"2024-09-18T01:00:21.926814+0000","last_unstale":"2024-09-18T01:01:26.029000+0000","last_undegraded":"2024-09-18T01:01:26.029000+0000","last_fullsized":"2024-09-18T01:01:26.029000+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'652","last_scrub_stamp":"2024-09-18T01:01:26.028766+0000","last_deep_scrub":"214'652","last_deep_scrub_stamp":"2024-09-18T01:01:26.028766+0000","last_clean_scrub_stamp":"2024-09-18T01:01:26.028766+0000","objects_scrubbed":14,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:16:17.475875+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.002915932,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6954,"num_read_kb":17130,"num_write":754,"num_write_kb":11285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":32804395,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"214'531","reported_seq":4856,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-18T01:01:31.324634+0000","last_change":"2024-09-18T01:01:31.324715+0000","last_active":"2024-09-18T01:01:31.324634+0000","last_peered":"2024-09-18T01:01:31.324634+0000","last_clean":"2024-09-18T01:01:31.324634+0000","last_became_active":"2024-09-18T01:01:01.242541+0000","last_became_peered":"2024-09-18T01:01:01.242541+0000","last_unstale":"2024-09-18T01:01:31.324634+0000","last_undegraded":"2024-09-18T01:01:31.324634+0000","last_fullsized":"2024-09-18T01:01:31.324634+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'531","last_scrub_stamp":"2024-09-18T01:01:31.324440+0000","last_deep_scrub":"214'531","last_deep_scrub_stamp":"2024-09-18T01:01:31.324440+0000","last_clean_scrub_stamp":"2024-09-18T01:01:31.324440+0000","objects_scrubbed":10,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:39.010518+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0027658140000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5114,"num_read_kb":26799,"num_write":585,"num_write_kb":11735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":9379374,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":482,"num_read":220594,"num_read_kb":1067291,"num_write":22261,"num_write_kb":688577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12013,"num_bytes_recovered":693846180,"num_keys_recovered":1158,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20024,"ondisk_log_size":20024,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509496,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51756,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320084,"statfs":{"total":96636764160,"available":96583766016,"internally_reserved":0,"allocated":2584576,"data_stored":1243311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509496,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52336,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319504,"statfs":{"total":96636764160,"available":96583172096,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509498,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60072,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311768,"statfs":{"total":96636764160,"available":96575250432,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509498,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55916,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315924,"statfs":{"total":96636764160,"available":96579506176,"internally_reserved":0,"allocated":2584576,"data_stored":1243311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509498,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60072,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311768,"statfs":{"total":96636764160,"available":96575250432,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509497,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52336,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319504,"statfs":{"total":96636764160,"available":96583172096,"internally_reserved":0,"allocated":3178496,"data_stored":1833679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509497,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47592,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324248,"statfs":{"total":96636764160,"available":96588029952,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51752,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320088,"statfs":{"total":96636764160,"available":96583770112,"internally_reserved":0,"allocated":2580480,"data_stored":1243292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:01:45.334 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-18T01:01:45.649 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-18T01:01:45.650 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-18T01:01:45.650 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T01:01:45.650 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:01:51.752 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T01:01:51.752 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T01:01:51.753 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:01:51.841 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T01:01:51.841 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-18T01:01:51.841 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:01:51.940 INFO:tasks.ceph.osd.2:Stopped 2024-09-18T01:01:51.940 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-18T01:01:51.940 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:01:52.074 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-18T01:01:57.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-18T01:01:57.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-18T01:01:58.042 INFO:tasks.ceph.osd.3:Stopped 2024-09-18T01:01:58.042 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-18T01:01:58.042 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:01:58.131 INFO:tasks.ceph.osd.4:Stopped 2024-09-18T01:01:58.132 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-18T01:01:58.132 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:01:58.201 INFO:tasks.ceph.osd.5:Stopped 2024-09-18T01:01:58.201 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-18T01:01:58.202 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:01:58.285 INFO:tasks.ceph.osd.6:Stopped 2024-09-18T01:01:58.285 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-18T01:01:58.285 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:02:03.487 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-18T01:02:03.487 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-18T01:02:04.386 INFO:tasks.ceph.osd.7:Stopped 2024-09-18T01:02:04.387 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-18T01:02:04.387 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-18T01:02:04.387 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:02:04.414 INFO:tasks.ceph.mgr.y:Stopped 2024-09-18T01:02:04.415 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-18T01:02:04.415 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:02:04.457 INFO:tasks.ceph.mgr.x:Stopped 2024-09-18T01:02:04.457 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-18T01:02:04.457 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-18T01:02:04.457 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:02:04.473 INFO:tasks.ceph.mon.a:Stopped 2024-09-18T01:02:04.474 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T01:02:04.474 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:02:04.532 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T01:02:04.532 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-18T01:02:04.532 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:02:04.548 INFO:tasks.ceph.mon.b:Stopped 2024-09-18T01:02:04.548 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-18T01:02:04.548 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-18T01:02:04.591 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-18T01:02:04.591 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-18T01:02:04.833 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-18T01:02:04.833 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-18T01:02:04.992 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-18T01:02:04.992 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-18T01:02:05.132 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-18T01:02:05.133 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-18T01:02:05.417 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-18T01:02:05.417 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-18T01:02:05.610 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-18T01:02:05.611 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-18T01:02:05.888 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-18T01:02:05.888 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-18T01:02:06.094 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-18T01:02:06.095 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-18T01:02:06.288 INFO:tasks.ceph:Archiving mon data... 2024-09-18T01:02:06.289 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680/data/mon.a.tgz 2024-09-18T01:02:06.289 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-18T01:02:06.295 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.PTeqrPig94 2024-09-18T01:02:06.295 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.PTeqrPig94 2024-09-18T01:02:06.451 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.PTeqrPig94 2024-09-18T01:02:06.515 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.PTeqrPig94 is 456KB 2024-09-18T01:02:06.576 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.PTeqrPig94 2024-09-18T01:02:06.581 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680/data/mon.c.tgz 2024-09-18T01:02:06.581 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-18T01:02:06.630 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.njSu97Buz3 2024-09-18T01:02:06.631 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.njSu97Buz3 2024-09-18T01:02:06.835 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.njSu97Buz3 2024-09-18T01:02:06.902 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.njSu97Buz3 is 614KB 2024-09-18T01:02:06.968 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.njSu97Buz3 2024-09-18T01:02:06.974 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680/data/mon.b.tgz 2024-09-18T01:02:06.974 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-09-18T01:02:06.980 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.bRb3tdoGLq 2024-09-18T01:02:06.980 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.bRb3tdoGLq 2024-09-18T01:02:07.168 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.bRb3tdoGLq 2024-09-18T01:02:07.232 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.bRb3tdoGLq is 614KB 2024-09-18T01:02:07.342 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.bRb3tdoGLq 2024-09-18T01:02:07.349 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-18T01:02:07.349 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T01:02:07.351 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T01:02:07.413 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-18T01:02:07.455 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-18T01:02:07.455 INFO:tasks.ceph:Archiving crash dumps... 2024-09-18T01:02:07.457 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680/remote/smithi042/crash 2024-09-18T01:02:07.458 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T01:02:07.472 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680/remote/smithi195/crash 2024-09-18T01:02:07.473 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T01:02:07.511 INFO:tasks.ceph:Compressing logs... 2024-09-18T01:02:07.511 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T01:02:07.519 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T01:02:07.539 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42928.log 2024-09-18T01:02:07.540 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34289.log 2024-09-18T01:02:07.541 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38136.log 2024-09-18T01:02:07.541 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42928.log.gz 2024-09-18T01:02:07.541 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34289.log.gz 2024-09-18T01:02:07.541 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40057.log 2024-09-18T01:02:07.544 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41673.log 2024-09-18T01:02:07.544 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38136.log: /var/log/ceph/ceph-client.admin.40057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38136.log.gz 2024-09-18T01:02:07.544 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35015.log 2024-09-18T01:02:07.545 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40057.log.gz 2024-09-18T01:02:07.545 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41673.log.gz 2024-09-18T01:02:07.545 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33529.log 2024-09-18T01:02:07.545 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32227.log 2024-09-18T01:02:07.545 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35015.log.gz 2024-09-18T01:02:07.546 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33529.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33529.log.gz 2024-09-18T01:02:07.546 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42616.log 2024-09-18T01:02:07.546 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32227.log.gz 2024-09-18T01:02:07.546 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31642.log 2024-09-18T01:02:07.581 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38247.log 2024-09-18T01:02:07.582 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42616.log.gz 2024-09-18T01:02:07.582 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31642.log.gz 2024-09-18T01:02:07.582 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-18T01:02:07.582 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42199.log 2024-09-18T01:02:07.583 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38247.log.gz 2024-09-18T01:02:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.34477.log 2024-09-18T01:02:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42199.log.gz 2024-09-18T01:02:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33401.log 2024-09-18T01:02:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34945.log 2024-09-18T01:02:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34477.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34477.log.gz 2024-09-18T01:02:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33401.log.gz 2024-09-18T01:02:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32602.log 2024-09-18T01:02:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.34945.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34552.log 2024-09-18T01:02:07.584 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34945.log.gz 2024-09-18T01:02:07.584 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32602.log.gz 2024-09-18T01:02:07.584 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43344.log 2024-09-18T01:02:07.584 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35810.log 2024-09-18T01:02:07.584 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34552.log.gz 2024-09-18T01:02:07.584 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41333.log 2024-09-18T01:02:07.584 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43344.log.gz 2024-09-18T01:02:07.584 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39603.log 2024-09-18T01:02:07.584 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35810.log.gz 2024-09-18T01:02:07.584 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34959.log 2024-09-18T01:02:07.585 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41333.log.gz 2024-09-18T01:02:07.585 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38981.log 2024-09-18T01:02:07.585 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39603.log.gz 2024-09-18T01:02:07.585 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32494.log 2024-09-18T01:02:07.585 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34959.log.gz 2024-09-18T01:02:07.585 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.38981.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43084.log 2024-09-18T01:02:07.585 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38981.log.gz 2024-09-18T01:02:07.585 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32494.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.41886.log 2024-09-18T01:02:07.585 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32494.log.gz 2024-09-18T01:02:07.586 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43084.log.gz 2024-09-18T01:02:07.586 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33504.log 2024-09-18T01:02:07.586 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41886.log: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-18T01:02:07.586 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42122.log 2024-09-18T01:02:07.586 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41886.log.gz 2024-09-18T01:02:07.586 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33504.log.gz 2024-09-18T01:02:07.586 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34206.log 2024-09-18T01:02:07.586 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42122.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-18T01:02:07.586 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42122.log.gz 2024-09-18T01:02:07.586 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37121.log 2024-09-18T01:02:07.586 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34206.log.gz 2024-09-18T01:02:07.587 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34785.log 2024-09-18T01:02:07.587 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39813.log 2024-09-18T01:02:07.587 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37121.log.gz 2024-09-18T01:02:07.587 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34785.log: /var/log/ceph/ceph-client.admin.42400.log 2024-09-18T01:02:07.587 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34785.log.gz 2024-09-18T01:02:07.587 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31818.log 2024-09-18T01:02:07.587 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39813.log.gz 2024-09-18T01:02:07.587 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42400.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42400.log.gz 2024-09-18T01:02:07.587 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29859.log 2024-09-18T01:02:07.587 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32010.log 2024-09-18T01:02:07.588 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31818.log.gz 2024-09-18T01:02:07.588 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.29859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32017.log 2024-09-18T01:02:07.588 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29859.log.gz 2024-09-18T01:02:07.588 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34625.log 2024-09-18T01:02:07.588 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32010.log.gz 2024-09-18T01:02:07.588 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33735.log 2024-09-18T01:02:07.588 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32017.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32017.log.gz 2024-09-18T01:02:07.588 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39875.log 2024-09-18T01:02:07.588 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34625.log.gz 2024-09-18T01:02:07.588 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33735.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33735.log.gz 2024-09-18T01:02:07.589 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41595.log 2024-09-18T01:02:07.589 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43318.log 2024-09-18T01:02:07.589 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39875.log.gz 2024-09-18T01:02:07.589 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41595.log.gz 2024-09-18T01:02:07.589 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40192.log 2024-09-18T01:02:07.589 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41078.log 2024-09-18T01:02:07.589 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43318.log.gz 2024-09-18T01:02:07.589 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41468.log 2024-09-18T01:02:07.589 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40192.log.gz 2024-09-18T01:02:07.590 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35627.log 2024-09-18T01:02:07.590 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41078.log.gz 2024-09-18T01:02:07.590 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39966.log 2024-09-18T01:02:07.590 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41468.log.gz 2024-09-18T01:02:07.590 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.35627.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38803.log 2024-09-18T01:02:07.590 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35627.log.gz 2024-09-18T01:02:07.590 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39114.log 2024-09-18T01:02:07.591 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39966.log.gz 2024-09-18T01:02:07.591 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42601.log 2024-09-18T01:02:07.591 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38803.log.gz 2024-09-18T01:02:07.591 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.39114.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40967.log 2024-09-18T01:02:07.591 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39114.log.gz 2024-09-18T01:02:07.591 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34683.log 2024-09-18T01:02:07.591 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42601.log.gz 2024-09-18T01:02:07.591 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40967.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.40967.log.gz --verbose 2024-09-18T01:02:07.591 INFO:teuthology.orchestra.run.smithi042.stderr: -- /var/log/ceph/ceph-client.admin.40193.log 2024-09-18T01:02:07.591 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31816.log 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34683.log.gz 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42043.log 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40193.log.gz 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33610.log 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31816.log.gz 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40265.log 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42043.log.gz 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42196.log 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33610.log.gz 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33261.log 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40265.log.gz 2024-09-18T01:02:07.592 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42196.log: /var/log/ceph/ceph-client.admin.40447.log 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42196.log.gz 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32224.log 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33261.log.gz 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31820.log 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40447.log.gz 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37444.log 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32224.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32224.log.gz 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40664.log 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31820.log.gz 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37444.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.37444.log.gz -- 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr: /var/log/ceph/ceph-client.admin.40306.log 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.40664.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32027.log 2024-09-18T01:02:07.593 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40664.log.gz 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40306.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40306.log.gz 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41621.log 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32027.log.gz 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35016.log 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.41621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34113.log 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41621.log.gz 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38638.log 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.35016.log.gz 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34113.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34113.log.gz 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39014.log 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40992.log 2024-09-18T01:02:07.594 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38638.log.gz 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39014.log.gz 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43110.log 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41469.log 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40992.log.gz 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43110.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43110.log.gz 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35958.log 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33348.log 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41469.log.gz 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35958.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35958.log.gz 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41157.log 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34261.log 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33348.log.gz 2024-09-18T01:02:07.595 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41157.log.gz 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36884.log 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43292.log 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34261.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34261.log.gz 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36884.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36884.log.gz 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34837.log 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33844.log 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43292.log.gz 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34837.log.gz 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42407.log 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32228.log 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33844.log.gz 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42407.log.gz 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37345.log 2024-09-18T01:02:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41517.log 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32228.log.gz 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37345.log.gz 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41782.log/var/log/ceph/ceph-client.admin.41517.log: 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41517.log.gz 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34161.log 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41782.log.gz 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38928.log 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42980.log 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34161.log.gz 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40522.log 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38928.log.gz 2024-09-18T01:02:07.597 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33140.log 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42980.log.gz 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40522.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40522.log.gz 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41912.log 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36938.log 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33140.log.gz 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40882.log 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41912.log.gz 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36938.log.gz 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31812.log 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34232.log 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40882.log.gz 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31812.log.gz 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35910.log 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41647.log 2024-09-18T01:02:07.598 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34232.log.gz 2024-09-18T01:02:07.599 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35910.log.gz 2024-09-18T01:02:07.599 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36383.log 2024-09-18T01:02:07.607 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26529.log 2024-09-18T01:02:07.607 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-18T01:02:07.607 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-18T01:02:07.607 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26385.log 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.7.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26529.log.gz 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26553.log 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr: 28.2% -- replaced with /var/log/ceph/ceph-client.admin.26385.log.gz 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26457.log 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26553.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26553.log.gz 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26457.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26457.log.gz 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26505.log 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26409.log 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26481.log 2024-09-18T01:02:07.608 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26505.log.gz 2024-09-18T01:02:07.609 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26409.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T01:02:07.609 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26409.log.gz 2024-09-18T01:02:07.609 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26433.log 2024-09-18T01:02:07.609 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26481.log.gz 2024-09-18T01:02:07.609 INFO:teuthology.orchestra.run.smithi195.stderr: 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T01:02:07.609 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-18T01:02:07.609 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.26433.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-18T01:02:07.609 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26433.log.gz 2024-09-18T01:02:07.609 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T01:02:07.651 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37632.log 2024-09-18T01:02:07.651 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41647.log.gz 2024-09-18T01:02:07.651 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36290.log 2024-09-18T01:02:07.651 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36383.log.gz 2024-09-18T01:02:07.651 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37632.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37632.log.gz 2024-09-18T01:02:07.651 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34726.log 2024-09-18T01:02:07.651 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38578.log 2024-09-18T01:02:07.651 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36290.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36290.log.gz 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39229.log 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34726.log.gz 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38578.log.gz 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35993.log 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42401.log 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39229.log.gz 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42096.log 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35993.log.gz 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42401.log.gz 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40853.log 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35785.log 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42096.log.gz 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37095.log 2024-09-18T01:02:07.652 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40853.log.gz 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35785.log.gz 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35318.log 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43006.log 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37095.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37095.log.gz 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32028.log 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35318.log.gz 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43006.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43006.log.gz 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41210.log 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33764.log 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32028.log.gz 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33807.log 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41210.log.gz 2024-09-18T01:02:07.653 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33764.log.gz 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41361.log 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37928.log 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33807.log.gz 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41361.log.gz 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34518.log 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37720.log 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37928.log.gz 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34518.log.gz 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35421.log 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.37720.log: /var/log/ceph/ceph-client.admin.42190.log 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37720.log.gz 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35421.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35421.log.gz 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43136.log 2024-09-18T01:02:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38761.log 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42190.log.gz 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43136.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.43136.log.gz -5 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42393.log 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43422.log 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38761.log.gz 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42393.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42393.log.gz 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32885.log 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43422.log.gz 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32946.log 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.32885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34907.log 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32885.log.gz 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36356.log 2024-09-18T01:02:07.655 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32946.log.gz 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34907.log.gz 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34430.log 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34003.log 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36356.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36356.log.gz 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35462.log 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34430.log.gz 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40133.log 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34003.log.gz 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36087.log 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35462.log.gz 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39760.log 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40133.log.gz 2024-09-18T01:02:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38395.log 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36087.log.gz 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31740.log 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39760.log.gz 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42410.log 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38395.log.gz 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36436.log 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31740.log.gz 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42410.log: /var/log/ceph/ceph-client.admin.42600.log 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42410.log.gz 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33451.log 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36436.log.gz 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32208.log 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42600.log.gz 2024-09-18T01:02:07.657 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38426.log 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33451.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33451.log.gz 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42824.log 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32208.log.gz 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38426.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38426.log.gz -5 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42590.log 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.42824.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41730.log 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42824.log.gz 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42590.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42590.log.gz 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39482.log 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41730.log.gz 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41437.log 2024-09-18T01:02:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36850.log 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39482.log.gz 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37637.log 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41437.log.gz 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36850.log.gz 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37395.log 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37512.log 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37637.log.gz 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37395.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41309.log 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37395.log.gz 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42148.log 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37512.log.gz 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41309.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41309.log.gz 2024-09-18T01:02:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40645.log 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40566.log 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42148.log.gz 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40815.log 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40645.log.gz 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32002.log 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40566.log.gz 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34752.log 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40815.log.gz 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32002.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32002.log.gz 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42615.log 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42069.log 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34752.log.gz 2024-09-18T01:02:07.660 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32520.log 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42615.log.gz 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42618.log 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42069.log.gz 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37516.log 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32520.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32520.log.gz 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42618.log.gz 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35674.log 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38164.log 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37516.log.gz 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41074.log 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35674.log.gz 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39396.log 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38164.log.gz 2024-09-18T01:02:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39201.log 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41074.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41074.log.gz 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39396.log.gz 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42189.log 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39201.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.39201.log.gz /var/log/ceph/ceph-client.admin.38369.log 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35346.log 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42189.log.gz 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35577.log 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38369.log.gz 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31714.log 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35346.log.gz 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35577.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35577.log.gz 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37209.log 2024-09-18T01:02:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34109.log 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31714.log.gz 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39305.log 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37209.log.gz 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37902.log 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34109.log.gz 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34629.log 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39305.log.gz 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37902.log.gz 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34629.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43396.log 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34629.log.gz 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32226.log 2024-09-18T01:02:07.663 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35101.log 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43396.log.gz 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32226.log.gz 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38536.log 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39578.log 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35101.log.gz 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38536.log.gz 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32644.log 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41990.log 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39578.log.gz 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32644.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32644.log.gz 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33894.log 2024-09-18T01:02:07.664 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41990.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39108.log 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41990.log.gz 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39255.log 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33894.log.gz 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42402.log 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37562.log 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39108.log.gz 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39255.log: /var/log/ceph/ceph-client.admin.42402.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39255.log.gz 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42402.log.gz 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39941.log 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38868.log 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37562.log.gz 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32416.log 2024-09-18T01:02:07.665 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39941.log.gz 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41387.log 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38868.log.gz 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32648.log 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32416.log.gz 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41387.log.gz 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35260.log 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31819.log 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32648.log.gz 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43162.log 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35260.log.gz 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33310.log 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31819.log.gz 2024-09-18T01:02:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34063.log 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43162.log.gz 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33310.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33310.log.gz 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41834.log 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32235.log 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34063.log.gz 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41543.log 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41834.log.gz 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41964.log 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32235.log.gz 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33078.log 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41543.log: /var/log/ceph/ceph-client.admin.41964.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41964.log.gz 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41543.log.gz 2024-09-18T01:02:07.667 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33078.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33078.log.gz 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32645.log 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36570.log 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35231.log 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32645.log.gz 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36570.log.gz 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38292.log 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41268.log 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35231.log.gz 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38292.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38292.log.gz 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38296.log 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42617.log 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41268.log.gz 2024-09-18T01:02:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38296.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38296.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.37203.log 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42617.log.gz 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36751.log 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39360.log 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37203.log.gz 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36751.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37765.log 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36751.log.gz 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39360.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39360.log.gz 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34325.log 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37304.log 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37765.log: /var/log/ceph/ceph-client.admin.34325.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37765.log.gz 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34325.log.gz 2024-09-18T01:02:07.669 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36121.log 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36038.log 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37304.log.gz 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36121.log.gz 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34365.log 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35155.log 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36038.log.gz 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34365.log.gz 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43266.log 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36564.log 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35155.log.gz 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43266.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40421.log 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36564.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36564.log.gz 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41704.log 2024-09-18T01:02:07.670 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40592.log 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40421.log: /var/log/ceph/ceph-client.admin.41704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40421.log.gz 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41704.log.gz 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40061.log 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41413.log 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40592.log.gz 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40061.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40061.log.gz 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32442.log 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41413.log.gz 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32984.log 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36498.log 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32442.log: /var/log/ceph/ceph-client.admin.32984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32442.log.gz 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32984.log.gz 2024-09-18T01:02:07.671 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33625.log 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43446.log 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36498.log.gz 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33625.log.gz 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40497.log 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43446.log.gz 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32020.log 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40497.log: /var/log/ceph/ceph.audit.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40497.log.gz 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38057.log 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32020.log.gz 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32007.log 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38057.log.gz 2024-09-18T01:02:07.672 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42016.log 2024-09-18T01:02:07.692 INFO:teuthology.orchestra.run.smithi195.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-18T01:02:07.697 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32007.log.gz 2024-09-18T01:02:07.697 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42202.log 2024-09-18T01:02:07.697 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42016.log.gz 2024-09-18T01:02:07.697 INFO:teuthology.orchestra.run.smithi042.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38495.log 2024-09-18T01:02:07.697 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.697 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42202.log.gz 2024-09-18T01:02:07.697 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37478.log 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38606.log 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38495.log.gz 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37478.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42200.log 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38606.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38606.log.gz 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33924.log 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35852.log 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42200.log.gz 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33924.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35852.log.gz 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36282.log 2024-09-18T01:02:07.698 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32572.log 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36282.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36282.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37761.log 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32572.log.gz 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40917.log 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43240.log 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37761.log.gz 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40917.log.gz 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32014.log 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43240.log.gz 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33970.log 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39681.log 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32014.log.gz 2024-09-18T01:02:07.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33970.log.gz 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42392.log 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39681.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39681.log.gz 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36539.log 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32220.log 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42392.log.gz 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36539.log.gz 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43032.log 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32220.log.gz 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32795.log 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32546.log 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43032.log.gz 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32795.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32795.log.gz 2024-09-18T01:02:07.700 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37999.log 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32546.log.gz 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42902.log 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42383.log 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37999.log.gz 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42902.log.gz 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39175.log 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42383.log.gz 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42602.log 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38001.log 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39175.log: /var/log/ceph/ceph-client.admin.42602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39175.log.gz 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42602.log.gz 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33040.log 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31815.log 2024-09-18T01:02:07.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38001.log.gz 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33040.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33040.log.gz 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37163.log 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31815.log.gz 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35724.log 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36680.log 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37163.log.gz 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35724.log.gz 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31669.log 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36680.log.gz 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43188.log 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42201.log 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31669.log.gz 2024-09-18T01:02:07.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43188.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26512.log 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43188.log.gz 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42201.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42201.log.gz 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31817.log 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42954.log 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26512.log.gz 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31817.log.gz 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33190.log 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42954.log.gz 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41569.log 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41938.log 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33190.log: /var/log/ceph/ceph-client.admin.41569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33190.log.gz 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41569.log.gz 2024-09-18T01:02:07.703 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34829.log 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41756.log 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41938.log.gz 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34829.log.gz 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42850.log 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41756.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41756.log.gz 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31803.log 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36981.log 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42850.log.gz 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31803.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.32635.log 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36981.log.gz 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39727.log 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32236.log 2024-09-18T01:02:07.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.32635.log: /var/log/ceph/ceph-client.admin.39727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39727.log.gz 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.32635.log.gz 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34393.log 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37011.log 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32236.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32236.log.gz 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.34393.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.34393.log.gz 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36704.log 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37011.log.gz 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39084.log 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43058.log 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36704.log.gz 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39084.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39441.log 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39084.log.gz 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43058.log.gz 2024-09-18T01:02:07.705 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36914.log 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40778.log 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39441.log.gz 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36914.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21399.log 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40778.log.gz 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42798.log 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41860.log 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.tmp-client.admin.21399.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21399.log.gz 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42798.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42798.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31694.log 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41860.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41860.log.gz 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32836.log 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26252.log 2024-09-18T01:02:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31694.log.gz 2024-09-18T01:02:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32836.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43214.log 2024-09-18T01:02:07.707 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26252.log.gz 2024-09-18T01:02:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35508.log 2024-09-18T01:02:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38720.log 2024-09-18T01:02:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43214.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43214.log.gz 2024-09-18T01:02:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35508.log.gz 2024-09-18T01:02:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40733.log 2024-09-18T01:02:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35603.log 2024-09-18T01:02:07.716 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38720.log.gz 2024-09-18T01:02:07.716 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38872.log 2024-09-18T01:02:07.716 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40733.log.gz 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35603.log.gz 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26407.log 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38872.log.gz 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36200.log 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32468.log 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.26407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26407.log.gz 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36200.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37849.log 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32468.log.gz 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33920.log 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35189.log 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37849.log: /var/log/ceph/ceph-client.admin.33920.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37849.log.gz 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33920.log.gz 2024-09-18T01:02:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36175.log 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38342.log 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35189.log: /var/log/ceph/ceph-client.admin.36175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35189.log.gz 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36175.log.gz 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38060.log 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35065.log 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38342.log.gz 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38060.log.gz 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33686.log 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.35065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35065.log.gz 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39916.log 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33166.log 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33686.log: /var/log/ceph/ceph-client.admin.39916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33686.log.gz 2024-09-18T01:02:07.718 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39916.log.gz 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31766.log 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36804.log 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33166.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33166.log.gz 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.31766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31766.log.gz 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42876.log 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40461.log 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36804.log: /var/log/ceph/ceph-client.admin.42876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42876.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36804.log.gz 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39516.log 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43370.log 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40461.log: /var/log/ceph/ceph-client.admin.39516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40461.log.gz 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39516.log.gz 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40352.log 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41808.log 2024-09-18T01:02:07.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43370.log: /var/log/ceph/ceph-client.admin.40352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43370.log.gz 2024-09-18T01:02:07.720 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40352.log.gz 2024-09-18T01:02:07.720 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41808.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41808.log.gz 2024-09-18T01:02:07.798 INFO:teuthology.orchestra.run.smithi042.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-18T01:02:07.799 INFO:teuthology.orchestra.run.smithi195.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-18T01:02:07.975 INFO:teuthology.orchestra.run.smithi042.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-18T01:02:08.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-18T01:02:08.619 INFO:teuthology.orchestra.run.smithi042.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-18T01:02:08.641 INFO:teuthology.orchestra.run.smithi195.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-18T01:02:09.909 INFO:teuthology.orchestra.run.smithi042.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-18T01:02:10.706 INFO:teuthology.orchestra.run.smithi195.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-18T01:02:11.016 INFO:teuthology.orchestra.run.smithi042.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-18T01:02:11.073 INFO:teuthology.orchestra.run.smithi042.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-18T01:02:11.073 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-18T01:02:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m3.553s 2024-09-18T01:02:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:user 0m10.740s 2024-09-18T01:02:11.074 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m0.488s 2024-09-18T01:02:11.407 INFO:teuthology.orchestra.run.smithi195.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-18T01:02:15.265 INFO:teuthology.orchestra.run.smithi195.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-18T01:02:15.267 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-09-18T01:02:15.267 INFO:teuthology.orchestra.run.smithi195.stderr:real 0m7.703s 2024-09-18T01:02:15.267 INFO:teuthology.orchestra.run.smithi195.stderr:user 0m15.361s 2024-09-18T01:02:15.267 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m0.650s 2024-09-18T01:02:15.267 INFO:tasks.ceph:Archiving logs... 2024-09-18T01:02:15.267 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680/remote/smithi042/log 2024-09-18T01:02:15.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T01:02:16.642 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680/remote/smithi195/log 2024-09-18T01:02:16.643 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T01:02:18.377 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-18T01:02:18.418 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-18T01:02:18.419 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T01:02:18.421 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T01:02:18.483 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-18T01:02:18.536 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-18T01:02:18.536 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T01:02:18.540 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T01:02:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T01:02:18.550 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================== 2024-09-18T01:02:18.551 INFO:teuthology.orchestra.run.smithi042.stdout:*hv01.front.sepi 67.205.162.81 3 u 56 64 177 0.072 +0.162 0.100 2024-09-18T01:02:18.551 INFO:teuthology.orchestra.run.smithi042.stdout:+hv02.front.sepi 63.231.80.2 3 u 58 64 177 0.078 +1.470 0.176 2024-09-18T01:02:18.551 INFO:teuthology.orchestra.run.smithi042.stdout:+hv03.front.sepi 74.6.168.72 3 u 58 64 177 0.094 -1.995 0.077 2024-09-18T01:02:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T01:02:18.558 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T01:02:18.558 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2024-09-18T01:02:18.558 INFO:teuthology.orchestra.run.smithi195.stdout:*hv01.front.sepi 67.205.162.81 3 u 55 64 177 0.065 -0.144 1.419 2024-09-18T01:02:18.558 INFO:teuthology.orchestra.run.smithi195.stdout: hv02.front.sepi 63.231.80.2 3 u 51 64 177 0.090 -0.110 1.712 2024-09-18T01:02:18.558 INFO:teuthology.orchestra.run.smithi195.stdout: hv03.front.sepi 74.6.168.72 3 u 54 64 177 0.087 -2.776 1.241 2024-09-18T01:02:18.559 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T01:02:18.559 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-18T01:02:18.576 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-18T01:02:18.577 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-18T01:02:18.605 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-18T01:02:18.615 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-18T01:02:18.624 INFO:teuthology.task.internal:Duration was 914.716074 seconds 2024-09-18T01:02:18.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-18T01:02:18.634 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-18T01:02:18.634 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T01:02:18.637 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T01:02:18.675 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-18T01:02:18.675 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-09-18T01:02:18.676 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T01:02:18.728 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-09-18T01:02:18.729 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T01:02:18.739 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-18T01:02:18.740 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T01:02:18.771 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T01:02:18.812 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-18T01:02:18.812 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T01:02:18.849 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T01:02:18.935 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-18T01:02:18.946 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-18T01:02:18.946 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T01:02:18.959 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T01:02:18.971 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-18T01:02:18.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:02:19.003 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:02:19.013 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-09-18T01:02:19.023 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-09-18T01:02:19.323 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:02:19.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:02:19.328 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:02:19.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:02:19.371 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-18T01:02:19.382 INFO:teuthology.task.internal:Transferring archived files... 2024-09-18T01:02:19.383 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680/remote/smithi042 2024-09-18T01:02:19.383 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T01:02:19.412 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909680/remote/smithi195 2024-09-18T01:02:19.412 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T01:02:19.435 INFO:teuthology.task.internal:Removing archive directory... 2024-09-18T01:02:19.435 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T01:02:19.447 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T01:02:19.488 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-18T01:02:19.500 INFO:teuthology.task.internal:Not uploading archives. 2024-09-18T01:02:19.500 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-18T01:02:19.510 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-18T01:02:19.510 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T01:02:19.512 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T01:02:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: 658468 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 01:02 /home/ubuntu/cephtest 2024-09-18T01:02:19.535 INFO:teuthology.orchestra.run.smithi195.stdout: 658616 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 01:02 /home/ubuntu/cephtest 2024-09-18T01:02:19.537 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-18T01:02:19.547 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-18T01:02:19.569 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read} duration: 914.7160744667053 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-18T01:02:19.570 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T01:02:19.644 INFO:teuthology.run:pass